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ABSTRACT 


The  restoration  of  images  and  the  enhancement  and 
detection  of  targets  in  cluttered  background  are  the  subjects 
of  this  research.   The  statistical  approach  is  used  in  order 
to  exploit  temporal  as  well  as  spatial  image  redundancies. 

The  images  are  modeled  as  a  homogeneous  random  field. 
An  autocorrelation  function  and  a  method  of  parameter 
identification  are  proposed.   Experiments  with  several 
pictures  are  presented  to  validate  the  model. 

An  analysis  of  two-dimensional  recursive  filters  is 
presented.   A  three-dimensional  recursive  filter  is  developed 
which  exploits  the  spatial  as  well  as  the  temporal  image 
redundancies . 

A  class  of  hybrid  filters  is  proposed  which  improves 
the  performance  of  the  recursive  filters.   Several  experi- 
ments with  pictures  are  presented  to  show  the  ability  of 
the  hybrid  filters  in  picture  restoration. 

A  detector  is  developed  for  purposes  of  target  extrac- 
tion from  cluttered  background  images.   The  detection  is 
independent  of  the  target  shape. 

A  simulation  of  the  target  detection  and  tracking 
problem  is  presented.   The  target  is  tracked  from  frame  to 
frame  by  means  of  a  conventional  Kalman  filter,  which  uses 
the  image  filter  as  the  measurement  device. 
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I.   INTRODUCTION 

A.   PROBLEM  DEFINITION 

This  research  involves  two  aspects  of  digital  image 
processing.   One  is  the  restoration  of  images  degraded  by 
white  Gaussian  observation  noise.   The  other  is  the  enhance- 
ment and  detection  of  targets  immersed  in  cluttered  back- 
ground images.   The  statistical  approach  is  used  in  order 
to  exploit  temporal  as  well  as  spatial  image  redundancies. 
The  emphasis  is  directed  towards  the  design  of  recursive 
and  hybrid  Bayesian  filters. 

In  the  recent  past  considerable  attention  has  been 
devoted  to  the  application  of  Kalman  filtering  to  smoothing 
of  observation  noise  in  image  data.   Habibi  [4]  first  sug- 
gested a  two-dimensional  recursive  image  estimator  as  an 
extension  of  the  one-dimensional  Kalman  filter.   Two  other 
similar  extensions  were  proposed  [3],  [5],   It  has  been 
shown  [10]  that  these  filters  do  not  preserve  the  optimality 
of  the  one-dimensional  Kalman  filter.   The  most  complete 
study  of  optimal  two-dimensional  Kalman  filtering  has  been 
performed  by  Woods  and  Radewan  [11] .   They  point  out  that  the 
generalization  of  the  Kalman  filter  to  two  dimensions  can 
be  done  optimally  with  an  extremely  high  dimensional  state 
vector,  which  has  dimensions  on  the  order  of  MN  (M  =  order 
of  the  filter,  N  =  width  of  the  image) .   Panda  and  Kak  [9] 
succeeded  in  deriving  a  vector  dynamic  model  that  generates 


the  same  random  field  of  Habibi's  model.   Since  this  model 
is  recursive  in  only  one  index,  the  one-dimensional  Kalman 
filter  is  applied  and,  of  course,  optimality  is  preserved. 
Again,  as  indicated  by  Woods  and  Radewan  [11] ,  an  extremely 
high  dimensional  state  vector  is  used,  which  has  the  same 
dimension  as  the  width  of  the  image. 

Several  techniques  have  been  investigated  for  background 
clutter  suppression  and  target  enhancement.   Statistical 
non-recursive  spatial  filters  [6]  are  suggested  for  background 
clutter  suppression  and  enhancement  of  targets  of  known  shapes. 
Two-dimensional  Kalman  filtering  [5]  is  also  proposed  for 
the  case  of  targets  with  arbitrary  shapes .   These  filters 
assume  that  the  image  gray  level  can  be  decomposed  into  three 
additive  components:   target,  background  and  white  observa- 
tion noise.   The  statistical  differences  of  these  three 
components  are  used  in  order  to  extract  the  target,  which 
is  the  desired  information. 

B.   RESEARCH  OBJECTIVES 

In  the  context  of  picture  restoration  and  target  detection 
as  defined  above,  several  specific  research  objectives  have 
been  identified. 

a)   Since  the  model  of  images  plays  a  fundamental  role 
in  the  statistical  approach  of  image  processing,  several 
experiments  with  real  life  pictures  will  be  accomplished 
in  order  to  validate  existing  models  [1-2]  and  to  suggest 
new  ones . 
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b)  Generalization  of  the  one-dimensional  Kalman  filter 
to  two  dimensions  results  in  excessive  computation  loads. 
On  the  other  hand,  the  filters  [3-51  are  very  simple  and, 
although  it  is  known  that  they  are  not  optimal,  it  was  not 
determined  yet  just  how  far  they  are  from  optimality.   There- 
fore, these  filters  will  be  compared  against  the  optimum 
non-recursive  filter. 

c)  The  existing  recursive  image  filters  exploit  only  the 
spatial  correlation,  therefore  a  three-dimensional  recursive 
filter  will  be  developed  in  order  to  take  advantage  of  the 
correlation  in  time . 

d)  The  possibility  of  improving  the  performance  of  the 
sub-optimum  recursive  filters  by  using  the  non-causal  (with 
respect  to  the  direction  of  recursion)  observations  closest 
to  the  estimated  pixel  (picture  element)  will  be  investi- 
gated.  The  result  is  a  hybrid  filter  in  the  sense  of  using 
some  observations  recursively  and  others  non-recursively . 

e)  An  optimum  decision  rule  will  be  derived  for  purposes 
of  background  suppression  and  target  enhancement  and  subse- 
quent threshold  detection. 

f)  A  conventional  Kalman  filter  will  be  constructed  to 
track  the  target  centroid  from  frame-to-frame. 

C .   OVERVIEW 

The  image  modeling  problem  is  discussed  in  Chapter  II. 
First,  a  tutorial  discussion  of  the  mathematical  problem  of 
finding  the  dynamic  model  of  random  fields  is  presented. 
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Second,  the  results  of  several  experiments  with  real  life 
pictures  are  presented.   Third,  a  model  for  a  picture  and  for 
pictures  sequenced  in  time  is  introduced. 

In  Chapter  III  the  sub-optimum  two-dimensional  recursive 
filters  [3-5]  are  analyzed.   The  exact  computation  of  the 
error  variance  is  particularly  important  to  those  filters, 
because  they  either  do  not  compute  the  error  variance  in 
the  calculation  of  the  gains,  or  use  an  approximation.   An 
exact  method  is  developed  to  compute  the  error  variance  of 
those  filters.   Using  such  a  method,  the  filters  are  compared 
among  themselves  and  against  the  optimum  non-recursive 
interpolator,  constrained  to  the  same  data  set.   A  recurs- 
ive filter  is  also  introduced  that  is  essentially  the 
same  as  [5],  but  the  computation  of  gains  is  accomplished 
without  approximation. 

In  Chapter  IV  a  three-dimensional  recursive  filter  is 
developed.   This  filter  estimates  the  pixel  gray  level  of 
pictures  sequenced  in  time  by  using  recursively  the  obser- 
vations of  the  estimated  frame,  as  well  as  those  in  the  pre- 
vious frames.   It  is  an  extension  of  the  two-dimensional 
recursive  filter  [3].   Numerical  results  are  presented  to 
evaluate  the  improvement  resulting  from  the  exploitation  of 
the  correlation  in  time. 

Chapter  V  introduces  a  new  class  of  image  filters, 
called  hybrid  filters.   These  filters  are  smoothers  that 
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combine  optimally  the  estimate  of  the  recursive  filters 
(two  or  three  dimensions)  with  an  arbitrary  set  of  "future" 
observations.   Theoretical  comparison  between  the  hybrid 
filter  and  the  recursive  [3]  and  non-recursive  [6]  filters 
is  accomplished.   Experiments  with  real  life  pictures 
are  also  presented  for  the  purpose  of  comparison  of  these 
filters. 

In  Chapter  VI  an  optimum  decision  rule  is  developed  to 
detect  targets  immersed  in  cluttered  background  images . 
The  target  is  considered  as  another  texture  statistically 
distinct  from  the  background  texture.   The  decision  is 
made  pixel  by  pixel  and,  therefore,  it  is  independent  of 
the  target  shape,  but  it  can  also  be  applied  to  targets  with 
known  shapes.   The  decision  is  based  on  the  observation  of 
the  pixel  gray  level  and  the  background  prediction  for  the 
pixel.   This  prediction  is  given  by  the  recursive  or  hybrid 
filters  and  may  also  be  given  by  a  non-recursive  filter. 
Some  special  cases  are  worked  out  and  its  performance 
evaluated.   The  image  is  modeled  as  a  weighted  addition  of 
three  components:   target,  background  and  observation  noise. 

In  Chapter  VII  a  conventional  Kalman  filter  is  constructed 
to  track  the  target  centroid  (or  other  points)  from  frame 
to  frame.   The  target  dynamics  in  the  picture  is  modeled.   The 
detector  developed  in  Chapter  VI  feeds  the  tracking  filter 
with  the  observations  of  the  centroid  spatial  coordinates. 
The  observation  error  is  analyzed. 
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Chapter  VIII  presents  several  results  of  the  simulation 
of  a  complete  target  detection  and  tracking  problem,  using 
computer  generated  images.   Both  recursive  and  hybrid 
filters  are  used  and  compared. 

The  final  chapter  summarizes  the  results  of  this 
investigation  and  presents  the  conclusions  and  suggestions 
for  further  study. 
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II.   IMAGE  MODELING 

A.  INTRODUCTION 

Our  objective  in  this  chapter  is  to  obtain  a  statisti- 
cal model  for  pictures.   Knowing  part  of  a  picture,  one 
can  generally  draw  certain  inferences  about  the  remainder; 
or,  knowing  a  sequence  of  frames,  one  can,  on  the  average, 
make  a  good  guess  or  prediction  about  the  next  frame.   From 
a  statistical  viewpoint,  similarity  between  adjacent  pixels 
(picture  elements),  or  ,  frame- to- frame  similarity  represent 
a  high  level  of  intraframe  or  interframe  correlation. 

Experimental  evicence  [l]-[2]  indicates  that  a  mono- 
chromatic image  can  be  modeled  by  specifying  its  value  (gray 
level  x(m,n)  at  each  spatial  coordinate  (m,n) .   An  ensemble 
of  such  images  can  be  modeled  by  interpreting  x(m,n)  as  a 
random  field. 

In  this  chapter,  first,  we  will  address  the  mathematical 
problem  of  finding  the  dynamic  model  of  random  fields,  given 
the  autocorrelation  function.   Second,  we  will  introduce  a 
model  for  a  picture  and  for  pictures  sequenced  in  time. 
Third,  we  will  present  experimental  results  in  order  to 
validate  the  proposed  model. 

B.  DYNAMIC  MODEL 

Assume  the  mean  and  autocorrelation  function  of  a 
homogeneous  (wide  sense  stationary)  random  field  are  given. 
Since  we  are  assuming  knowledge  of  the  mean,  for  convenience, 
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we  assume  the  random  field  has  zero  mean,  therefore  auto- 
correlation and  autocovariance  are  the  same. 

A  highly  desirable  characteristic  of  any  dynamic  model 
is  to  have  it  excited  by  uncorrelated  noise.   The  impulse 
response  method  always  has  such  characteristics.   The  diffi- 
culty with  this  method  is  that  the  power  spectrum  density 
has  to  be  factored,  and  there  is  no  mathematical  theorem 
for  factorization  in  the  multi-dimensional  case.   Since 
there  exists  considerable  evidence,  in  refs.  [l]-[2]  and 
in  this  research,  that  images  are  well  modeled  by  auto- 
correlation functions  with  separable  kernels,  the  difficulty 
of  factorization  will  not  arise  and,  therefore,  the  impulse 
response  method  will  be  used. 

The  impulse  response  method  is  shown  in  figure  2.1. 
Assume  there  exists  a  system  transfer  function  H(z),  such 
that,  when  driven  by  white  noise  W(n),  the  resultant  output 
X(n)  is  a  random  process  with  the  desired  autocorrelation 
function  R(k) .   Using  input-output  relationships  between 
power  spectrum  densities  (PSD) : 


SQ(z)   =   Si(z)H(z)H(z"1] 


Since  the  input  is  white  noise,  its  PSD  is  constant, 
Si(z)  =  K,  thus: 


H(z)H(z_1)   =   |sQ(z)   =   iz(R(k)) 
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w(n) 
Si(z) 


X(n) 
So(z) 


Figure  2.1 
Filter  Response  Method 

Therefore,  to  compute  H(z)  we  have  to  factor  the 
right  hand  side  of  the  previous  equation.   This  is  always 
possible,  in  the  one  dimensional  case,  provided  S  (z)  is 
a  ratio  of  polynomials. 

In  the  remainder  of  this  section  we  present  some  exam- 
ples to  illustrate  the  method,  as  well  as  for  later  use. 
Example  1 

Consider  the  autocorrelation  function: 


^  ii  s  2   -a  k 

R(k)  =  a   e   ' 


k  =  0,±1,±2, 


let 


P   =   e 


-a 


R(k)   ■   a2  p'k 


(2.1) 


Take  the  two-sided  Z-transform  of  R(k) , 
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Z(R(k))   =     I     R(k)z 
k=-°° 


-k 


2        22  -12-2 

=  a    ( .  .  .  +  p  z  +  pz  +  l+pz   +pz   +...) 


2     2 

Z(R(k))   =     °  (17P  } (2.2) 

(1-pz  x) (1-pz) 


S.(z)H(z)H(z_1)   =     g2(1"p2) 


1  '  (i-pz_1) (i-pz: 


To  have  a  stable  model  the  poles  of  H(z)  must  be  inside 
the  unit  circle,  therefore  we  choose: 


H(z)   =   -1      -   X(Z) 


,    "I      W(z) 
1-pz 


Si(z)   =   a2(l-p2: 


Thus ,  the  dynamic  model  is . 


X(n)   =   pX(n-l)  +  W(n)  (2.3) 


Initializing  equation  (2.3)  with  X(0),  and  exciting 

2     2 
with  white  noise,  having  variance  a  (1-p  ) ,  a  random  process 

X(n)  is  generated  having  the  desired  autocorrelation 

function. 
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Example  2 

Consider  the  autocorrelation  function: 

2  ,  , 

0,n   _  a r,,   2,   k  +1  ,,   2,   k  +1. 

R(k)   "   (p1~p2)(l+p1p2)[(1-P2)Pl     -d-P1)P2l  '   1 

(2.4) 

where : 

-a,  -a- 

P-L   =   e     ,p2   =   e",k  =  0,±1,±2,  ... 

Using  results  of  example  1,  the  Z-transform  of  R(k)  is: 


Z(R(k))   = Q 


1-P1z   ) (l-p2z   ) (l-p1z) (l-p2z; 


(2.5) 


where : 


2/i      » 
a  (l-p,p7) 

Q   "     1+Plp2   d-Pi  )d-P22) 


Let's  choose: 


H(z)   =   X(Z) 


W(z)       ,,     -1,  ,,     -1, 
(1-P12    )  (1_P2Z    ) 


si(z: 


Thus,  the  dynamic  model  is 
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X(n)   =   (p1+p2)X(n-l)  -  p1P2X(n-2)  +W(n)     (2.6) 


It  can  be  easily  shown  that  a  particular  case  of  this 
model,  when  p,  approaches  p2,  is: 


R(k)   =   a2  p'k' (1 +  6  |k| )  (2.7) 


where 


1  +  P 
and  the  dynamic  model  is: 


_      1  -  p  -a 

3   = ^2    i         p   =   e 


X(n)   =   2pX(n-l)  -  p2X(n-2)  +  W(N)  (2.8) 


2 /i     2,3 
S  (Z)   =   °  (1  -  V 

l  +  o 


From  equation  (2.7)  we  can  see  that  this  autocorrela- 
tion function  has  zero  derivative  at  the  origin  (k  =  0) 
and  an  inflection  point  at  k  =  1/a ,  provided  that  p  >  0.7 
or  3  ~  ct. 

The  model  given  by  equation  (2.4)  is,  therefore,  a  more 
general  case  and  includes,  as  particular  cases,  the  models 
given  by  equations  (2.1)  and  (2.7).   In  figure  2.2  some 
curves  of  equation  (2.4)  are  shown  for  the  same  "correlation 
time"  (  the  point  where  the  correlation  is  37%  of  the 
maximum) . 
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correlation  time  =  10 
(1)   P.  =  0 


Figure  2.2 
Autocorrelation  Functions 
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Example  3 

Consider  a  two-dimensional  autocorrelation  function, 
separable  in  the  two  dimensions: 


R(i,j)   =   a2  pv|x|  pj3 I  (2.9) 


where 


Pv  =   e    ,   ph   =   e    ,   x,j  =  0,±1, 


To  find  the  Z-transform  we  take  advantage  of  separa- 
bility and  use  equation  (2.2)  from  example  1. 

2      2       2 
a  (1-p/)  U-Ph  ) 

Z(R(i,j))   =   ^—t- S_ 

(1-Pvz1   )  d-Phz2   )  ^"^l5  (1-phz2) 

(2.10. 

Let's  choose: 


1  X(z  ,z  ) 

H(z, ,z_) 


'12'      ,,      -1.  ,,      -1,      W(z,  ,z~) 
(1~PVZ1   } (1_phZ2   )         1      2 


Si(z1,z2)   =   a2(l-pv2) (l-ph2) 


Thus,  the  dynamic  model  is 


X(m,n)   =   p  X(m-l,n)+phX(m,n-l)-p  phX(m-l,n-l)+W(m,n) 


(2.11) 
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Example  4 

Similar  to  the  previous  example,  consider  a  separable 
autocorrelation  with  the  kernels  of  example  2 : 


R(i,j)   =   a2  Kv(i)  Kh(j)  (2.12) 


where: 


tt      i  •  a                                                 1  r  ,  -,  2-  1     +1       ...  2,  1+1, 

K        l)         =        -, TTTT rld-Pi  Pi  ~(l-p,  )  P-,      '       '  ] 

v         (p,  -p„  ) (1+p,  p„       K2v   Klv  lv   M2v 

lv   2v      lv  2v 


K,(j)   =   same  equation  with  i  +      j 

v   -*•   h 


The  Z-transform  is  given  by: 


00  00 


_1   z.-3 


Z(R(i,j))       =  I  I       R(i,j)     Z1         ,.., 

j_  =  — co      -i=  — co 


CO  CO 


O2        I      i      I     Kh(j)z2~D]Kv(k)21~1 

i  =  —  CO       "j=  — CO 


a2    Z1(Kv(i))     Z2(Kh(j)) 


Using  equation  (2.5)  it  can  be  easily  verified  that  the 
transfer  function  is: 
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H(Z.,,Z2)    -  ^  —  3^  -s— 

(1-Plvz1i)(l-P2v2l  1)d-Plhz2  1)(l-p2hz2  X) 

(2.13) 


Si(z1,z2)   =   a2  Qv  Qh 


where : 


(1"plvp2v}       2       2 

Qv   =   rr-£Hr(1-piv  }  (1_p2v  : 

lv  2v 


Q.   =   same  equation  with  v   ■>  h 


Thus,  the  dynamic  model  is: 


X(m,n)   =   (p.  +p„  )X(m-l,n)  -  p..  p0  X(m-2,nl 
Mlv   2v  lv  2v 


+  (plh+P2h)X(m,n-l)   -  plhp2hX(m,n-2! 


(Plv+p2v^(plh+p2h)X(m'1'n_1) 


+  plvp2v(plh+p2h)X(m_2'n-1) 


+  plhP2h(plh+p2v)X(m~1'n-2) 


~  plvp2vplhp2hX(m_2,n~2)  +  w(m'n)  (2.14) 

Observe  that  example  3  is  a  particular  case  of  this, 
where  p2v  =  p2h  =  0. 
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To  simplify  notation,  from  now  on,  we  will  adopt  the 
following  convention: 


X(m,n)   =   A  X  +  W(m,n) 


(2.15) 


where: 


A  =   row  vector  of  coefficients 


X  =   column  vector  of  adjacents 


In  the  case  of  example  3: 


A   = 


[p   p,   -p  p  ' 
v  h    v  h" 


X  = 


X(m-1  ,n) 
X(m,n-1) 
X(m-l,n-i; 


Example  5 

Consider  a  three-dimensional  separable  autocorrelation 
function  having  kernels  like  examples  1  and  2 : 


R(i,j,k)   =   a   Kx(k)  K2(i,j 


2.16 


where: 


Kx(k)   =   p 
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K2(i,j)   =   Kv(i)Kh(j)   (see  example  4) 

Following  similar  procedures  as  in  example  4,  it  can 
be  verified  that  the  transfer  function  is: 


H(z, , z-» Z-)  — 


(1-plvZr1}  (1-p2vZr1}  (1-plhZ2_1)  (1-p2hZ2_1)  (i-ptz3_1) 


(2.17) 


Si(zl'Z2'Z3)   =  °2    QvQh  (1"pt2) 


where  0   and  Q,  are  the  same  as  in  example  4. 
Thus,  the  dynamic  model  is: 

X(m,n,t)   =   A  X  +  W(m,n,t)  (2.18) 

where : 

A  is  a  row  vector  whose  elements  are  the  coefficients 
of  eq.  2.14,  followed  by  these  same  coefficients 
multiplied  by  p  ,  and  the  final  element  is  p  . 

X   is  a  column  vector  whose  elements  are  the  adjacent 
pixels  of  eq.  2.14,  followed  by  these  same  pixels 
located  on  the  previous  frame,  and  the  final  element 
is  X(m,n,t-1) . 

In  figure  2.3  the  adjacent  pixels  are  shown. 
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n-2     n-1       n 


-*-  ^ 


n-2     n-1      n 


ift-2 

m-1 
m 


m-2 
m-1 

m 


TIME  =  t  -  1 


TIME  =  t 


Figure  2.3 
Adjacent   pixels  of  example  5 

C.   AN  AUTO- REGRESSIVE  MODEL 

In  this  section  we  present  a  stochastic  model  for 
images.   The  basic  assumption  is  that  images  constitute  a 
homogeneous  random  field  with  zero-mean  (or  known  mean) 
and  known  autocorrelation  function,  separable  in  the  inde- 
pendent  dimensions.   These  assumptions  will  be  validated 
by  experiments  in  the  next  section. 

1.   Autocorrelation  Function 

The  autocorrelation  function  chosen  is  intended  to  be 
general  enough  to  include  most  of  the  models  used  in  other 
research   [2]  -  [9]  ,  as  well  as  to  best  fit  the  experimental 
functions  measured  in  this  research  and  [l]-[2].   The 
hypothesis  of  separability  allows  us  to  examine  the  auto- 
correlation kernel  by  kernel.   Assume  that  the  process  is 
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modeled  by  a  kernel  of  first  order,  as  in  Example  1 
The  dynamic  model  is 


X(n)   =   pjXCn-1)  +  W1(n) 


If  this  model  is  valid,  the  modeling  error  W,  (n)  must  be 
uncorrelated  noise.   This  can  be  easily  verified  by  computing 
the  autocorrelation  of  W, (n)  as  follows.   We  compute  W, (n) 
at  each  point  by 


W1(n)   =   X(n)  -  p1X(n-l) 


and  then  compute  the  autocorrelation  of  this  sequence. 
Computing  the  autocorrelation  of  W, (n)  is  a  very  good 
test.   Assume  that  W, (n)  turns  out  to  be  correlated. 
In  this  case  we  can  model  W, (n)  again  by  a  first  order 
kernel: 

Wx(n)   =   p2W1(n-l)  +  W2(n) 

Assume  that,  after  repeating  the  same  measurements  for 
W~ (n)  ,  we  conclude  that  W„ (n)  is  uncorrelated  noise. 
In  figure  2.4  it  is  shown,  in  the  Z-domain,  the  operations 
that  we  have  performed. 

Thus,  the  overall  transfer  function  is: 
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w2(z) 


X(z) 


Figure    2.4 
Second-order   model 


H(z)       - 


_      X(z) 


w^TzT 


(i-p1z~  )  d-p2z"  ; 


The  second-order  dynamic  model  is 


X(n)   ■   (p1+p2)X(n-l)  -  p1P2X(n-2)  +  W2 (n)        (2.19) 


We  have  now  a  model  driven  by  white  noise,  which  is  a 
second-order  difference  equation. 

The  process  generated  by  this  model  has  an  autocorrela- 
tion function  given  by  equation  (2.4),  as  we  will  demonstrate 


E(W2  (n)  )   =   Q2 
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Z(R(k))       =      Q2    H(z)     H(z    X) 


where : 


since 


Q2 


(1-Pj^z       )  (1-p    z)  d-P2z       )  (1-P2Z) 


AQ2  BQ2 


d-P1z"1)  (i-p]_z)        (i-p2z-1)  (i-p2z; 


(P1"P2)  (1-P1P2) 


B      = 


"p2 


(p.-pT)  (1-p1p  2) 


z      ( ? )      =     j  p  ' 

(l-pz    x)  (1-pz)  1-p^ 


t-i/i\                 s\     r       **                   k                   B  k     , 

R(k)       =      Q     [- J   pi  - 2    pi      I] 

1-P]_  1-P2 


2 

Using   R(0)    =   a       (variance   of   X(n) 


2  2  2       (1_PiP2) 

Q2      =      a2(l-Pl2)(l-p22)     (1+plp2}  (2.20) 


R(k)       =       (P1-P2M1+P1p2)[pl(1-P22)pl|k|-P2(1-Pl2)p2!k^ 

(2.21) 
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Assume,  now,  that  W„ (n)  is  not  uncorrelated 
enough.  In  such  a  case  we  proceed  to  a  third-order  model. 
Following  similar  procedures  as  for  the  second-order  model, 
we  have : 

Dynamic  Model: 


X(n)   =   (p1+p2+p3)X(n-l)  -  (p1p2+p1P3+p2P3)X(n-2) 


+  Plp2p3X(n-3)  +W3(n)  (2.22! 


Autocorrelation  function: 


1 3c  I    ,    kl    _    Ikl 

-  A, p  '   '  +  A~p~ '   '  +  A  p  '   ' 

R^       -  A1    +    l2\    A3 LJ—  (2-23: 


where: 


2  2      2 

Al   =   Pl  ^P2~P3^  (-p2p3) (1-p2  '   (1-p3  ' 


2  2       2 

A2   =   -p2  (p1-p3)  (l-p1P3)  (l-px  )  (1-P3  ) 


2  2       2 

A3   =   p3  ^pl~p2^  ^1-plp2^  ^1-pi  ^  ^1-p2  ' 


Variance  of  W_ (n) : 


i_  +  = +  £ 

2-,    2    .    2 

1-PX    l-p2     l-p3 


31 


The  general  expressions  for  a  k-order  model  are: 
Dynamic  model: 
X(n)  =  (p.+p2+ pk)X(n-l)  -  (p1p2  +  p1p3+.  .  .p]<_1pk)  X(n-2) 


+  (PlP2P3+...pk_2Pk_lPk)X(n-3) 

+  (-l)k+1PlP2...pkX(n-k)  +  Wk(n)  (2.25) 


Autocorrelation  function: 


,  A  p  I1'  +  A-P-  I1'  +  ...  A  p  I1! 

«<«    -    °        H  +  *2*  ■■■  *\ ^~~    (2'26) 


Variance  of  W,  (n)  : 


2 

Qk   =   "A A2 A"  (2'27) 

1        2   +  k 


,    2  '      2 2 

1-P]_     1~P2  l"Pk 


The  expressions  for  the  A's  are  easily  inferred 
from  equation  (2.23). 

In  example  3  an  auto-regressive  model  is  presented 
for  a  picture  with  an  autocorrelation  whose  kernels 
in  the  two  spatial  dimensions  are  the  first  order  versions 
(k=l)  of  equation  (2.26).   In  fact,  that  model  is  used  in 
most  research   [21  -  [9]  with  images.   In  example  4 
an  image  model  is  presented  whose  kernels  are  the  second- 
order  versions  (k=2)  of  equation  (2.26).   The  order  of  the 
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kernels  doesn't  have  to  be  the  same.   Depending  on  the 
particular  picture  to  be  modeled,  the  order  is  chosen  for 
each  direction. 

In  example  5  an  auto-regressive  model  is  presented 
for  pictures  sequenced  in  time.   In  that  case  we  chose 
second-order  kernels  in  the  spatial  directions  and  first- 
order  in  the  time  direction. 

At  this  point,  some  special  features  of  the  proposed 
autocorrelation  functions  given  by  equation  (2.26)  should 
be  emphasized: 

a)  It  includes  as  a  particular  case  the  first 
order  model  which  is  used  in  most  research  [2]  -  [9] . 

b)  It  fits  much  better  many  situations  where  the 
first  order  model  is  a  poor  approximation. 

c)  The  resultant  dynamic  model  is  auto-regressive 
and  driven  by  white  noise. 

d)  The  cascade  feature  of  whitening  the  modeling 
error  is  quite  simple  and  adequate  for  on  line  parameter 
identification.   This  feature  will  be  exploited  in  the  next 
item. 

2 .   Parameter   Identification 

Given  that  the  autocorrelation  function  of  an 
image,  or  a  time-frame  of  images,  is  assumed  to  have 
separable  kernels  as  in  equation  (2.26),  the  next  problem 
is  to  find  its  parameters.   Let's  discuss  the  modeling  of 
just  a  picture,  since  the  extension  for  the  time-frame  is 
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straightforward.   Having  a  specific  picture,  we  want  to 
identify  the  parameters  of  the  model.   To  compute  statistics 
from  just  one  realization  of  the  process  we  must  assume 
the  process  is  ergodic,  therefore  we  consider  the  picture 
as  a  typical  realization  and  the  averaging  in  space  is 
equivalent  to  statistical  averaging.   Also  the  size  of  the 
picture  has  to  be  sufficiently  large. 

Those  conditions  are  quite  severe,  and  we  realize 
that  the  process  has  to  be  restricted  to  a  particular 
class  of  pictures  with  common  properties.   Another  way 
of  looking  at  this  is  to  visualize  the  picture  as  an  array, 
for  example,  of  100x100,  therefore  with  10,000  pixels. 
Each  pixel  can  have,  say,  100  distinguishable  brightness 
values.   The  number  of  different  pictures  is  100   ' 
With  such  enormous  numbers  of  pictures,  even  if  we  were 
able  to  compute  the  mean  and  autocorrelation,  such  moments 
would  not  be  enough  information.   It  would  be  necessary 
to  know  statistics  of  enormously  higher  orders.   The  atten- 
tion, therefore,  has  to  be  focused  on  local  redundancy, 
encompassing  only  a  few  pictures  closely  related.   To  have 
a  model  as  accurate  as  possible,  based  only  in  the  first 
and  second  moments,  we  will  need  some  degree  of  adaptation. 

Our  method  is  based  on  the  cascade  feature  of 
equation  (2.26).  Let's  explain  the  method  in  detail. 
Assume  a  picture  is  given  with  negligible  measurement 
noise.   We  proceed  in  the  following  steps. 
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Step  1  -  Compute  the  mean  and  remove  it  from  the  picture. 
Step  2  -  Compute  the  autocorrelation  in  the  horizontal 
direction,  with  a  number  of  displacements  equal  to  20%  of 
the  number  of  columns.   There  is  no  use  to  go  much  further 
than  this,  because  the  accuracy  in  the  averaging  is  reduced. 
Repeat  the  same  procedure  in  the  vertical  direction.   This 
step  can  be  accomplished  using  the  Fast  Fourier  Transform 
algorithm  (FFT) . 

Step  3  -  Having  the  measured  autocorrelation  in  the  hori- 
zontal and  vertical  directions,  assume  that  equation  (2.26) 
is  adequate  and  gives  a  nice  fit.   Start  with  a  first-order 
kernel  and  compute  p.   and  p.,  ,  such  that  the  mean-squared 
error  in  the  fitting  is  minimized.   The  result  is  a  model 
like  example  3: 


X(m,n)   =   A   X  +  W,(m,n) 


where; 


— 1        lv   Klh     lv^lh 


XT   =   [X(m-l,n)   X(m,n-1)   X(m-l,n-l)] 


Step  4  -  At  this  point  we  have  a  model  for  the  picture  that 
is  first-order  in  both  spatial  directions.   To  test  the 
goodness  of  this  model  the  obvious  figure  of  merit  is  the 
modeling  error  W, (m,n) .   If  the  model  is  adequate,  W, (m,n) 
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should  be  uncorrelated  noise.   To  test  this,  compute  the 
autocorrelation  of  W,  (m,n)  as  in  step  2.   If  it  is  corre- 
lated in  both  directions,  it  means  the  first-order  model  is 
not  good  enough  and  we  have  to  proceed  to  a  second-order  model . 
It  could  happen  that  we  are  satisfied  with  one  direction, 
but  not  with  the  other,  in  this  case  we  have  to  proceed 
to  a  second-order  only  in  such  directions.   Assume  here 
that  W, (m,n)  is  correlated  in  both  directions. 


Step  5  -  Repeat  step  3  for  W,  (m,n)  in  order  to  find  p 
and  P?h-   The  model  for  W. (m,n)  is: 


where ; 


W1(m,n)   =   A2  W.  +   W2(m,n) 


A„   =   [p_    p„,   -p„  p~.  ] 
—2        2v    2h    p2v^2h 


-1T   =   [W^m-l,!!)   W1(m,n-1)   W1(m-l,n-l) 


2v 


The  second-order  model  for  the  picture  is  given  by 
equation  (2.14)  in  example  4. 

Step  6  -  Repeat  step  4  for  W~(m,n).  If  it  is  sufficiently 
uncorrelated,  the  procedure  stops  and  we  have  identified 
the  order  of  the  model  as  well  as  the  parameters.   Other- 
wise we  proceed  seeking  a  higher  order  model. 
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NOTE:   Step  4  is  responsible  for  the  decision  to  proceed 
or  not  and  also  in  which  directions,  whether  horizontal, 
vertical  or  both.   The  decision  to  proceed  or  not  can  be 
implemented  only  based  on  the  variance  of  W(m,n),  by  observing 
its  reduction  after  the  second-order  modeling.  If  such  reduc- 
tion is  null  or  negligible,  it  means  the  original  model 
is  good  and  it  is  not  necessary  to  increase  the  order.   The 
decision  about  the  direction  to  be  improved  can  be  imple- 
mented by  observing  the  autocorrelation  of  W(m,n)  at  a  few 
points,  say,  the  first  5  displacements. 

The  computation  of  the  coefficients  is  quite  simple 
because  we  have  reduced  the  problem  to  fitting  exponentials 
at  each  step.   Such  computation  is  accomplished  as  follows. 
Assume  the  measured  autocorrelation  in  one  direction  is 
given  by  Y(n),  n  =  0,1,... K  and  it  is  normalized  such  that 
Y(0)  =  1.   We  want  to  fit  an  exponential  minimizing  the 
mean-squared  error,  R(n)  =  e    =  p  .   Taking  the  natural 
logarithm  of  R(n)  and  Y(n) ,  we  want  to  minimize: 


K 


z2     =        I    [In  R(n)  -  In  Y(n) ]2 
n=l 


K 
ez  =    I     [-an  -  In  Y(n) ]2 
n=l 
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Take  the  derivative  with  respect  to  a   and  equate 
to  zero,  the  result  being: 

K  K 

I    n  In  Y(n)    -6   J  n  In  Y(n) 
n=l  n=l 


a   = 


K  K(K+1) (2K+1) 

n=l 


and 


-a 
p   =   e   . 


In  figure  2.5  a  flowchart  to  implement  the  proposed 
method  of  modeling  is  shown. 

To  model  pictures  sequenced  in  time  the  procedure 
is  quite  similar.   In  this  case  we  have  a  sequence  of  pic- 
tures, sequenced  in  time,  and  apply  the  method  adding  the 
extra  dimension  (time) .   Example  5  presents  a  model  that  is 
first-order  in  time  and  second-order  in  both  spatial 
dimensions . 

3 .   Modeling  Of  Noisy  Images 

In  the  last  item,  a  method  for  picture  modeling  was 
presented  in  which  the  picture  was  assumed  to  be  noise  free. 
Such  an  assumption  restricts  considerably  real  life  appli- 
cations where  there  is  no  knowledge  of  the  original  picture, 
This  restriction  can  be  removed  if  the  autocorrelation 
function  of  the  measurement  noise  is  known. 


38 


START 


COMPUTE  MEAN  AND 
VARIANCE  OF  THE  PICTURE 
REMOVE  THE  MEAN 


COMPUTE  HORIZONTAL  AND 
VERTICAL  CORRELATIONS 


COMPUTE: 
°lv  and  Olh 


COMPUTE:   wjm.n) 
and  its  mean  and 
variance.  Remove  mean. 


i=2 


COMPUTE  VERTICAL 
CORRELATION 


YES     /UNCC 

R.     \ 

\     m         / 

°iv  ■   ° 

\VERT?  / 

|NO 

COMPUTE    0iv 

COMPUTE    HORIZONTAL 
CORRELATION 

YES/UNOC 

i          / 

KLZ7/ 

*ih    =    ° 

\hoi 

[no 

COMPUTE    o . . 

in 

COMPUTE:   wMm.n) 
and  its  mean  and 
variance .  Remove  mean . 


!  NO 


Figure  2.5 
Parameter  Identification  Flowchart 
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In  what  follows,  we  present  the  steps  to  be  modi- 
fied in  the  previous  modeling  method,  in  order  to  take  into 
account  additive  zero-mean  white  measurement  noise,  with 
known  variance. 

The  measurement  equation  is : 

y(m,n)   =   X(m,n)  +  v(m,n) 

where  v(m,n)  is  zero-mean  white  noise,  uncorrelated  with 
X(m,n)  and  with  known  variance: 


2 
E (v  ( m,n) )   =   r 


Step  2  -  Compute  the  autocorrelation  of  the  noisy  picture 

<y(m,n)  in  both  directions,  as  before,  but  remove  the 

variance  R  from  R.   (0,0)  to  obtain  Rv  (0,0),  since: 

yy  A^i. 


Ryy(0,0)  -  r         (i,j)   =   (0,0) 


Rxx ( i ,  j  )   =  { 


Rvv(i,j)  (i,j)  *      (0,0) 


In  figure  2.6  the  relation  between  the  autocorrela- 
tions of  X(m,n)  and  y(m,n)  is  illustrated  in  one  direction. 
Step  4  -  Since  X(m,n)  is  not  known,  W. (m,n)  can't  be  calcu- 
lated, but  its  autocorrelation  can  be  computed  indirectly. 
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Figure  2.6 
Autocorrelation  of  y(m,n) 


X(m,n)   =   Ax  X  +  W1(m,n) 


(2.28) 


y(m,n)   =   X(m,n)  +  v(m,n) 


(2.29) 


Substituting  (2.28)  into  (2.29): 


(m,n)   =   A,  Y  +  v(m,n)  +  W-^n^n)  -  A_x  V 


where: 


YT  =   [y(m-l,n)   y(m,n-l)   y(m-l,n-l)] 


V 


T   =   [v(m-l,n)   v(m,n-l)   v(m-l,n-l)] 
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Let's  call: 


v,  (m,n)   =   v(m,n)  -  A-,  V  (2.30) 


u,  (m,n)   =   W,  (m,n)  +  v, (m,n) 


y(m,n)   =  A-  Y  +  u,(m,n)  (2.31) 

Since  y(m,n)  and  A,  are  known,  u, (m,n)  can  be 
computed,  and,  therefore,  its  autocorrelation  in  both 
directions. 

Let's  examine  the  autocorrelation  of  u, (m,n)  in 
the  horizontal  direction. 

E  [u1 (m,n) u  (m,n+j ) ]  =  E [v,  (m,n) v,  (m,n+j ) ]  +  E[W,  (m,n) W,  (m,n+j) ] 

E[v,  (m,n) v,  (m,n+j ) ]  =  E [v (m,n) v (m,n+ j ) ]  -  A, E [v (m,n) v (m,n+j )  ] 


-  A, E [v (m,n+j ) v (m,n) ] 


+  A  E  [v(m,n)  vT(m,n+j  )  ]  A.T 


where: 


E[v(m,n)  v(m,n+j  )  ]   =  r5(0,j) 
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E[v(m,n) v(m,n+j ) ]      = 


0 
r<5(0,j-l) 

0 


E[v(m,n+j  )  v(m,n)  ]      = 


r6  (0,j+l) 
0 


E [v (m,n) v    (m,n+j ) ] 


r6(0,j)  0  r<5(0,j-l) 

0  *6(0,j)  0 

rS(0,j  +  l)       0  ro(0,j) 


Therefore: 


RViVi(0,j)      =      r(l+Piv2+Plh2+Plv2plh2)5(0'^ 

-  rPlh(l+Plv2)6(0,j-l)  -  rplh(l+plv2)5(0,j  +  l) 


luu<0,j)        -      ^v^0'*'    +   Vl(°'jl 


or: 


\^0'^ 


^u/^^-^^lv'^lh^Plv^lh2^'^    =    ° 


=  ^u^0'"11    -  rplh(1+plv   » 


j    -  1 


R  ( 0  ,  j )     elsewhere 

u,  u..         J 
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In  figure  2.7  the  relation  between  the  auto- 
correlations of  u, (m,n)  and  W,  (m,n)  is  illustrated. 


^R     (0,j 
ul'ul 


R    (0,j) 
wl'wl 


Figure  2.7 
Autocorrelation  of  u, (m,n) 


The  procedure  above  is  easily  extended  to  calculate 
the  autocorrelation  of  W2(m,n),  as  well  as  higher  order 
modeling  errors. 


w 


,  (m,n)   =   A2  W_x  +  W2(m,n) 


(2.32! 
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u, (m,n)   =  W, (m,n)  +  v, (m,n)  (2.33) 


Substitute  (2.32)  into  (2.33) 


u,(m,n)   =  A,  U.  +  u2(m,n)  (2.34) 


where : 


u„ (m,n)   =   W2(m,n)  +  v2(m,n)  (2.35] 


v2(m,n)   =  v-^da,]!)  -  A2  V, 


From  equation  (2.34)  we  can  calculate  u2(m,n), 
since  u,(m,n)  was  computed  by  equation  (2.31),  and  the 
coefficients  A„  are  given  by  the  fitting  of  R-.  „  (i,j)  . 

To  continue  the  modeling,  the  autocorrelation 

of  W_(m,n)  is  derived  from  R     (i,j)  using  equation  (2.35). 

U2U2 

D.   EXPERIMENTAL  RESULTS 

In  this  section  we  present  some  relevant  results  obtained 

with  real  life  pictures.   Several  pictures  were  analyzed 

and  divided  in  two  broad  classes: 

Picture  A  -  represents  a  class  of  pictures  with  few 

details  (i.e.  a  low  content  of  high  spatial  frequency 
structure) .   In  figure  2.8  a  sample  of  such  pictures  in 
a  three-dimensional  plot  is  shown,  where  the  height  (z-axis) 
is  the  gray  level.   This  kind  of  plot  is  very  useful  to 
emphasize  small  details. 
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Picture  B  -  represents  a  class  of  pictures  with  many- 
details.   Figure  2.14  shows  a  sample  of  such  pictures. 

1.   Autocorrelation  Function 

The  proposed  model  assumes  that  pictures  have 
separable  autocorrelation  functions  whose  kernels  are  given 
by  equation  (2.26).   To  validate  this  assumption,  the 
autocorrelation  of  pictures  A  and  B  were  measured  and 
compared  with  that  of  the  proposed  model.   Figure  2.9  is 
the  measured  autocorrelation  of  picture  A,  and  figure  2.15 
is  the  same  for  picture  B.   Using  minimum  mean-squared 
error  criteria,  a  first-order  model  was  fitted  for  both 
pictures,  as  shown  in  figures  2.10  and  2.16.   Such  a  model 
didn't  work  well  for  picture  A,  as  can  be  seen,  comparing 
figures  2.9  and  2.10.   For  picture  B  it  worked  quite  well, 
as  can  be  seen  comparing  figures  2.15  and  2.16.   Figure  2.11 
is  the  second-order  model  for  picture  A;  observe  that  it  is 
a  very  good  fit.   Based  on  these  results,  we  conclude: 

a)  The  proposed  autocorrelation  is  adequate  for 
picture  modeling. 

b)  The  first-order  model  used  in  other  research 
does  not  fit  well  pictures  of  class  A. 

c)  Pictures  with  few  details  are  best  fitted  by 
a  second-order  model. 

d)  Pictures  with  many  details  are  adequately 
fitted  by  first-order  model. 
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2 .  Parameter   Identification 

The  proposed  method  of  parameter  identification 
was  applied  to  both  pictures.   Figures  2.12  and  2.17  are  the 
autocorrelation  of  W1fm,n)  for  pictures  A  and  B,  respectively. 
The  autocorrelation  of  the  modeling  error  W  (m,n)  is  the 
measurement  of  the  "goodness "-of  the  first-order  model.  As 
predicted  before,  picture  A  has  a  poor  first-order  model,  since 
W, (m,n)  is  quite  correlated,  instead  of  white  noise,  as 
can  be  seen  in  figure  2.12.   On  the  other  hand,  picture  B 
has  a  nice  first-order  model,  since  W, (m,n)  is  almost  white 
noise,  as  can  be  seen  in  figure  2.17. 

Applying  the  cascade  method  of  modeling  for  picture 
A,  the  second-order  modeling  error  W-(m,n)  was  calculated 
and  its  autocorrelation  plotted  in  figure  2.13.   Now  the 
modeling  error  W„(m,n)  is  quite  close  to  white  noise,  and 
the  second-order  model  is  adequate  for  picture  A,  as 
predicted  before. 

3 .  Bandwidth  Compression 

In  order  to  present  another  piece  of  evidence  to  validate 
the  model,  it  was  applied  to  bandwidth  compression.   Picture  B 
is  quantized  in  256  gray  levels,  thus,  8  bits  are  required 
to  transmit  the  gray  level  of  each  pixel.   Exploiting 
redundancies  in  the  picture  and/or  in  a  time  frame  of 
pictures  (like  television) ,  it  seems  possible  to  reduce 
the  number  of  bits/pixels  to  be  transmitted,  therefore, 
reducing  the  required  bandwidth. 
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The  model  for  picture  B  is: 


X(m,n)   =   A  X  +  W.(m,n) 


The  measured  variances  of  X(m,n)  and  W, (m,n)  are 


VAR  (X)   =   4  9 


VAR  (W1)  =    1 


Also,  it  was  seen  that  W, (m,n)  is  almost  uncorre- 
lated  noise.   Therefore,  instead  of  transmitting  the  picture 
X,  we  can  transmit  W,  : 

W,(m,n)   =   X(m,n)  -AX 

Since  W,  is  close  to  white  noise,  and  has  much 
smaller  variance  than  X,  the  quantization  levels  for  W  may 
be  much  smaller  than  for  X.   This  method  of  bandwidth 
reduction  is  called  DPCM  (Differential  Pulse-code  Modulation; 

Of  course,  the  effectiveness  of  this  method  is 
strongly  dependent  on  the  "goodness"  of  the  model. 

Figure  2.18  presents  the  histogram  of  the  modeling 
error  W, .   It  can  be  seen  that  most  "energy"  of  W,  is 
concentrated  around  zero.   In  figure  2.19  the  reconstruction 
of  picture  B  is  presented,  where  W,  was  transmitted  with 
only  2  bits/pixel  (average) ,  instead  of  the  8  bits/pixel 


48 


required  for  X.   Therefore,  even  with  a  bandwidth  reduction 
of  4  times,  the  reconstructed  picture  yet  carries  most  of 
the  information  of  the  original. 
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Figure    2.8 
Picture   A 
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Figure  2.9 
Measured  Autocorrelation  of  Picture  A 
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Figure  2.10 
First  Order  Model  of  Picture  A 
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Figure  2.11 
Second  Order  Model  of  Picture  A 
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Figure  2.12 
First  Order  Modeling  Error  of  Picture  A 
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Figure  2.13 
Second  Order  Modeling  Error  of  Picture  A 
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Figure    2.14 
Picture   B 
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Figure  2.15 
Measured  Autocorrelation  of  Picture  B 
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Figure  2.16 
First  Order  Model  of  Picture  B 
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Figure  2.17 
First  Order  Modeling  Error  of  Picture  B 


59 


X 

X 


X 


X 
X* 


X 


X 


in 

C\J 

o 


X 


2* 


X 


IT) 

o 


^ 


U5 

o 

X 


X 
X 


X 


X 


X 


002 


000 


Figure    2.18 
Histogram  of   First   Order  Modeling 
Error    of    Picture    B 
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Figure  2.19 
Reconstruction  of  Picture  B  with 
Bandwidth  Compressed  4:1 
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III.   TWO-DIMENSIONAL  RECURSIVE  FILTERS 

A.   INTRODUCTION 

A  wide-sense  Markov  (WSM)  sequence,  corrupted  by  addi- 
tive white  noise,  can  be  optimally  (in  the  linear  least 
squares  sense)  estimated  by  a  recursive  estimator.   The 
fact  that  image  intensity  can  be  modeled  by  a  generaliza- 
tion of  a  WSM  sequence  (Chapter  II)  motivates  one  to  search 
for  a  two-dimensional  Kalman  estimator  to  smooth  additive 
white  observation  noise  in  images.   In  references  [3],  [4] 
and  [5]  three  filters  are  proposed.   These  filters  use  the 
dynamic  model  in  equation  (3.2),  that  is  a  particular  case 
(first  order  WSM)  of  the  more  general  model  proposed  in 
Chapter  II.   The  structure  of  these  filters  is  also  the 
same  (see  equation  3.4),  the  difference  being  the  method  of 
gain  computation.   The  reason  of  the  existence  of  three 
Bayesian  filters  with  the  same  structure,  but  different 
gains,  is  that  they  are  all  sub-optimum.   It  has  been 
determined  [10]  that  the  optimality  of  the  Kalman  filter 
is  not  preserved  when  generalized  to  two  dimensions,  with 
such  dynamic  models.   In  reference  [9]   is  proposed 
a  vector  model,  described  by  a  first-order  linear  n-dimensional 
vector  difference  equation,  that  is  recursive  in  one  parameter 
and  generates  the  same  random  field  used  in  [3-5] .   With 
such  a  model  the  Kalman  filter  is  applied  preserving,  of 
course,  optimality.   Unfortunately,  the  complexity  of 
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implementation  of  such  a  filter  is  enormous,  due  to  the 
size  of  the  state  vector,  that  is  the  number  of  pixels  in 
one  row,  say,  256  or  even  100.   Also,  its  complexity  increases 
drastically  if  we  need  to  extend  it  to  three-dimensional 
random  fields,  as  in  the  case  of  time-frame,  which  will  be 
addressed  in  chapter  IV.   Similarly,  the  extension  to  higher 
order  WSM  random  fields  increases  its  complexity. 

Because  of  the  above  considerations,  we  will  examine 
more  carefully  the  sub-optimum  filters  in  [3-5],  due  to  their 
simplicity  and  adequacy  to  accommodate  an  extra  dimen- 
sion (time)  or  a  higher  order  WSM  random  fields. 

In  this  chapter  we  will  compare  the  three  filters 
[3-5]  among  themselves,  as  well  as  against  the  optimum 
non-recursive  interpolator,  constrained  to  the  same  data 
set.   We  will  also  introduce  a  recursive  filter  that  is 
essentially  the  same  as  [5],  but  the  computation  of  gains 
is  accomplished  without  approximation.   The  figure  of 
merit  will  be  the  error  variance,  since  this  is  the  cost 
function  used  by  all  three  filters.   Observe  that  filters 
[4]  and  [5]  compute  error  variance  in  order  to  calculate 
the  gains,  but  there  is  an  approximation  in  the  recursive 
equation,  and  this  may  result  in  big  errors  in  the  gains, 
as  well  as  in  the  computed  error  variance. 

In  the  remainder  of  this  chapter,  we  will  develop  an 
algorithm  to  compute  the  variance  of  the  estimation  error, 
in  order  to  compare  the  filters  having  the  structure  of 
equation  (3.4)  and  dynamic  model  of  equation  (3.2)  .   Since, 
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filters  [4]  and  [5]  have  similar  algorithms  for  computation 
of  gains,  and,  because  filter  [4]  is  clearly  inferior  to 
[5],  only  the  former  will  be  used  in  the  comparison. 

B.   ALGORITHM  FOR  COMPUTATION  OF  ERROR  VARIANCE 

Given  the  dynamic  model,  the  filter  structure  and  the 

gain,  we  are  going  to  develop  an  algorithm  to  compute  the 

variance  of  the  estimation  error. 
1.   Dynamic  Model 

The  picture  is  modeled  by  a  zero-mean  random  field, 

homogeneous,  with  autocorrelation  function: 


R(i,j)   =   a2  pj1'  pjj  j  I  (3.1: 


The  dynamic  model  is : 

X(m,n)  =  pvX  (m-l,n)+p,X  (m,n-l) -p  p,X  (m-1  ,n-l)  +W  (m,n) 

(3.2) 
where  the  gray  level  X(m,n)  has  the  autocorrelation  of 
equation  (3.1)  and  W(m,n)  is  the  random  forcing  input  (or 
modeling  error).   W(m,n)  is  zero-mean  white  noise,  uncorre- 
cted with  X(p,q),  for  all  pixels  (p,q)  in  region  X 
defined  in  figure  3.1. 


E(W2(m,n))   =   a2(l-pv2)  d-Ph2)   =   Q         (3.3) 
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Figure  3.1 
Definition  of  the  region  X 
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2.   Filter  Structure 

The  filters  [3,5]  use  equation: 


X(m,n)   =   [1-K(m,n) ] [pvX(m-l,n)  +  phX(m,n-i; 


-  pvphX(m-l,n-l) ]  +  K(m,n)y(m,n)  (3.4) 

where  y(m,n)  is  the  measurement  of  X(m,n) : 

y(m,n)   =   X(m,n)  +  v(m,n)  (3.5) 

where  v(m,n)  is  zero-mean  additive  white  measurement  noise, 
uncorrelated  with  X(m,n),  and  having  variance: 


2 

E  (v  (m,n)  )   =   r 


The  estimate  X(m,n)  can  be  seen  as  the  optimal 

linear  combination  of  the  prediction  X  (m,n) ,  and  the 

P 

measurement  y (m,n) : 


X(m,n)   =   [l-K(m,n)]X  (m,n)  +  K(m,n)y(m,n)  (3.6) 

hr 


where : 


X  (m,n)   =   pvX(m-l,n)  +  phX(m,n-l)  -  PvPh^ (m-l,n-l) 


(3.7) 
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The  prediction  X  (m,n)  carries  all  the  information 
about  X(m,n),  contained  in  the  "past"  measurements.   The 
gain  K(m,n)  is  computed  such  that  all  the  information  about 

X(m,n),  contained  in  X   (or  the  "past"  measurements)  and 

P 

y(m,n),  is  effectively  used.   If  that  happens,  the  estima- 
tion error 


e(m,n)   =   X(m,n)  -  X(m,n) 

is  uncorrelated  with  the  measurements  used  to  estimate 

X(m,n)  . 

It  is  shown  [10]  that  the  estimation  error  can  not 

be  orthogonal  to  all  the  "past"  measurements,  but  may  be 

only  to  the  region  Y   ,    ,,  defined  in  figure  3.2,  which 
J  3     m-l,n-l  3 

excludes  row  m  and  column  n. 

As  a  matter  of  fact,  filters  [3-5]  do  not  have 

estimation  error  orthogonal  to  all  measurements  in  Y   ,    . 

3  m-l,n-l 

although  incorrectly  stated  in  [3]  and  [4].   A  simple  way  to 
prove  this  above  is  using  a  counterexample.   Let's  examine 
the  estimation  of  X(2,3).   According  to  [3]  and  [4],  the 
estimation  error  of  this  pixel  should  be  orthogonal  to 
yd,  2).   Also  the  error  in  the  estimation  of  X(2,2)  can't 
be  orthogonal  to  yd,  2),  according  to  [10].   We  are  going 
to  verify  that  these  statements  are  incompatible. 
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The  error  in  the  estimation  of  x(2,3)  is 

e(2,3)   =   X(2,3)  -  X(2,3) 

Using  equations  (3.2),  (3.4)  and  the  fact  that 
W(2,3)  and  v(2,3)  are  uncorrelated  with  yd, 2),  it  can  be 
verified  that 


e(2,3)y(l,2)   -   [1-K (2, 3) [pve (1, 3) y (1,2) 


+  phe(2,2)y(l,2)  -  PyPhe ( 1 , 2) y (1 , 2) ] 


Since  the  filters  reduce  to  the  one-dimensional 
Kalman  filter  for  the  first  row: 


e(l,3)y(l,2)   =   e(l,2)y(l,2)   =   0 


Thus 


e(2,3)y(l,2)   =   [1-K (2 , 3) ] phe (2 , 2) y (1 , 2) 

Therefore,  since  in  general  K(2,3)  ^  1  and  e(2,2)  can  not 
be  orthogonal  to  y(l,2),  we  conclude  that  e(2,3)  is  not 
orthogonal  to  y(l,2). 

3.   Variance  of  the  Estimation  Error 

In  the  following  derivation  we  will  simplify 
notation  by  dropping  the  argument  (m,n) ,  where  this  does 
not  cause  confusion. 

The  variance  of  the  estimation  error  is  given  by: 
P(m,n)   =   E[(X-X)2]   =   E(X2)  +  E(X2)  -  2E  (XX) 

6  8  


Let's  call: 


~2         2 

E(XZ)   =  OqZ 


E(XX)   =   b 


P(m,n)   =  a2   +  a  2  -  2b  (3.8: 


a.   Variance  of  the  Estimate 

2 

To  compute  o   ,  observe  that  X  can  be  written 

as  a  linear  combination  of  all  measurements  in  region 

Y     (see  figure  3.2). 
m,n 


X(m,n)   =  b11Y(l,l)  +  ...  b±    Y(l,q)  +  •••  blny(l,n)  + 


bplY(p,l)  +  ...  bpqY(p,q)  +  ...  bpny(p,n)  + 


bmly(m,l)  +  ...  bY(m,q)    +  ...  b   Y(m,n) 
ml  mq  mn 


(3.9) 


Define  the  column  vectors : 


T 
B   =   [b,  t   b.  _  . . .  b  t  .  .  .  b 

—        11    12      pi      mn' 


YT  =   [y(l,D   yd, 2)  ...  y(p,l)  ...  y(m,n)] 
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Substituting  in  (3.9) 


X(m,n)   =   BY 


(3.10! 


The  estimate  variance  is 


or 


2        ~2        T     T 
a     =   E(X^)   =   B  E(YY  )B 


2       T 
a    =   B  R,v  B 

e       —        i  — 


(3.11) 


where : 


RY   =   E(YYX) 


Using  equations  (3.1)  and  (3.5),  it  can  be 
easily  seen  that  the  elements  of  the  autocorrelation  matrix 
Ry  are  given  by: 


R(k-p,£-q)  (k,£)  ?     (p,q) 


RY(i,j)   =  < 


(3.12: 


a      +   r 


(k,£)  =  (p,q) 


where  i  is  the  sequential  number  for  the  pairs  (k,2,)  and 
j  for  (p,q)  as  follows 
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(k,£)  and  (p,q)   =   (1 , 1)  ,  (1 ,  2)  , . . .  (m,n) 
i  and  j   =   1,2,  ...  mn 

b.   Covariance  of  X  and  X 

Using  equation  (3.10),  the  covariance  b  is 
given  by: 


b   =   E(XX)   =   BTE(XY! 


or 


b   =   BTRxy  (3.13: 


where 


RXY   =   E(X-) 


The  elements  of  the  vector  R-,y.  are  given  by: 
RxY(i)   =   R(m-k,n-£)  (3.14; 

where  i  is  the  sequential  number  for  the  pairs  (k,£)  with 
(k,£)   =   (1,1) , (1,2) ,... (m,n) 
i   =   1,2,3,.. .mn 
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4 .   Calculation  of  the  Coefficients 

To  find  the  vector  of  coefficients  B,  observe  in 
equation  (3.9)  that  when: 


Y(i,j)   =  < 


(i,j)   =   (p,q) 


(i/j)   7*   (P/q) 


(3.15) 


The  result  is: 


X(m,n)   =   b 


pq 


It  can  be  easily  verified  that,  using  (3.15) 
and  (3.4) ,  all  coefficients  can  be  computed.   By  means  of 
a  simple  computer  program,  therefore,  we  can  compute  the 
exact  weight  that  any  measurement  has  in  the  estimation  of 
X. 

Let's  summarize  the  steps  to  compute  the  error 
variance  at  (m,n) : 

(1)  To  find  each  coefficient  b  use  equation  (3.4) 
with  the  values  of  y  given  by  (3.15).  The  value  of  X(m,n) 
will  be  b    .   The  initial  conditions  are: 

P/q 

X(i,j)   =   0    for   i  =  (p-1)  or  j  =  (q-1) 

2 

(2)  Use  equations  (3.11)  and  (3.12)  to  compute  a 
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(3)  Use  equations  (3.13)  and  (3.14)  to  compute  b. 

(4)  Compute  error  variance  by  (3.8)  . 

C.   ANOTHER  RECURSIVE  FILTER 

In  this  section  we  will  introduce  another  sub-optimum 
recursive  filter  having  the  same  structure  as  those  in  [3-5] 
It  is  similar  to  [5] ,  but  it  computes  the  gains  without 
approximations . 

The  objective  is  to  find  a  way  to  calculate  the  gains, 
such  that  the  variance  of  the  estimation  error  is  minimized. 

Let's  call  the  covariance,  between  X  and  its  prediction 

2 
X  ,  b  ;  and  the  variance  of  the  prediction  a 
p   p  *  p 

Using  equation  (3.6): 


b   =   E(XX)   -   (l-K)E(XX  )  +  KE(Xy) 


but: 


bp  =  E(xxp; 


E(Xy)   =   E(X2)   =  a2 


Thus: 


b   =   (l-K)b   +  Ka2  (3.16) 

P 


Squaring  equation  (3.6)  and  taking  the  expected 
value: 
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E(X2)   =   (1-K)2E(X  2)  +  K2E(y2)  +  2K(l-K)E(yX  ) 

P  P 


but: 


2        ~  2 

«p2  -'  E'V: 


2        2 

ECy  )  =  u  +  r 


E(yX  )   =   E(XX  )   =   b 

p       p     p 


Therefore: 


a  2   =   (1-K)2a  2+K2(a2+r)+2K(l-K)b  (3.17) 

e  P  p 


Substituting  equations  (3.16)  and  (3.17)  in  (3 


P(m,n)   =   (l-K)2(a2+a  2-2b  )  +  K2r  (3.18 


2 
Observe  in  (3.18)  that  a  "    and  b   are  independent  of 

P       P 

the  gain  K(m,n),  but  they  are  functions  of  "past"  gains. 

2 
Therefore  we  can  find  K(m,n),  as  a  function  of  a      ,    b  , 

P    P 
2 
a   and  r,  such  that  the  error  variance  is  minimized. 

Differentiating  (3.18)  with  respect  to  K  and  equating 

to  zero: 

2      2 

a      +   o        -    2b 

K(m,n)   =  — ^2 ~ (3-19: 

a   +  a    -  2b   +  r 
P      P 
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The  variance  of  the  prediction  error  is : 


P  (m,n)   =   E[(X-X  )2]   ■  a2    +    a    2    -    2b      (3.20) 
p  P  P       P 


Substituting  in  (3.19; 


K(m,n)   =   p — f—  (3.21) 

P 


Equation  (3.20)  is  intuitively  appealing,  because  it  is 
the  same  as  that  of  the  one  dimension  scalar  Kalman  filter. 
Substituting  (3.19)  into  (3.18): 


PMIN   =   Kr  (3.22) 


Therefore,  with  the  gain  calculated  by  (3.21),  the 

minimum  error  variance  is  given  by  (3.22). 

Observe  that  P  (m,n)  is  independent  of  K(m,n) ,  and, 
y 

therefore,  can  be  computed  using  "past"  gains.   Its 
calculation  is  quite  similar  to  that  of  P(m,n). 


T 
X  (m,n)   =   AY  (3.23) 

P  P 


where : 


T  .      - 

—        11    12      pi      m,n-l 


X„T  =   [y(l,U  yd, 2)   ...  y(p,l)  ...  y(m,n-l)] 
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The  variance  of  X   is 

P 


a  2   =   ATR7  A  (3.24) 


where  Ry   is  the  autocorrelation  matrix  of  the  measurements 
P 

V 

The  covariance  b  (m,n) ,  similarly  to  b(m,n),  is  given 


by: 


P 


b    =  h*XY  (3.25! 


where  the  vector  R.y   is  the  correlation  between  X  and  Y 

P  l 

The  coefficients  A  are  calculated  identically  to  B, 

but  using  the  predictor  equation  (3.7)  instead  of  (3.4). 

A  summary  of  gain  calculation   follows : 

(a)   Use  equations  (3.15)  and  (3.7)  to  find  the 

coefficients  A.   The  initial  conditions  for  (3.7)  are: 


X(i,j)   =   0    for   i  =  (p-1)   and   j  =  (q-1) 


2 

(b)  Use  equation  (3.24)  to  find  a 

(c)  Use  equation  (3.25)  to  find  b  . 

(d)  Use  equation  (3.20)  to  find  P  ,  and  compute 
K(m,n)  by  (3.21) . 


D.   NON- RECURSIVE  FILTER 

The  optimum  recursive  filter  must  have  the  same 
performance  of  the  non-recursive  Wiener  filter,  provided 
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the  data  set  is  the  same.   Since  two-dimensional  recursive 
filters,  like  those  in  [3-5],  are  not  optimum,  we  want  to 
find  how  good  they  are. 

The  minimum-mean-square-error  (MMSE)  in  the  estimation 
of  X(m,n),  using  the  measurements  in  region  Y     (see 
figure  3.2),  is  computed  as  follows. 

Define  the  vector 


Y   = 


y(i,D 

y(i,n) 
y(2,i) 

y (m,n) 


H   = 


h 


h 


h 


1,1 


l,n 

2,1 


m,n 


The  estimate  of  X(m,n)  is 


X(m,n)   =   H  Y 


and  the  MSE 


Let; 


P(m,n)   =   E[(X-X)2]   =   E[(X  -  HTY)2] 


2        2 

E(XZ)   =   a 


11 


E(YYT)   =   R^ 


E(XY)   =   Rxy 


P(m,n)   =   o2  -  2HTRXY  +  HTRyYH  (3.26) 


Differentiate  (3.26)  and  equate  to  zero: 


H  =   Ryy"1  Rxy  (3.27) 


Substitute  (3.27)  into  (3.26! 


PMIN(m,n)  =      a2    -   HTRxy  (3.28) 


It  can  be  shown  that  when  the  observation  follows  (3.5) 
the  MMSE  is  given  by 


PMT.T(m,n)   ■  h    r  (3.29) 

MIN  m,n 


E.   PERFORMANCE  OF  THE  FILTERS 

In  this  section  we  present  the  results  of  the  comparison 
between  filters  [3],  [5]  and  the  one  introduced  in  Section  C, 
as  well  as  the  optimum  interpolator  of  Section  D. 

First  of  all,  we  have  compared  filters  [5]  and  that  of 
Section  C,  since  they  are  essentially  the  same,  but  the 
former  uses  an  approximation.   This  comparison  was  accom- 
plished by  computing  the  gain  and  error  variance  of  both 
filters  for  several  situations.   The  result  was  that 
they  presented  practically  the  same  values;  therefore 
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we  concluded  that  the  approximation  made  in  [5]  is  very 
good.   This  conclusion  is  important,  because  it  allows  the 
use  of  simple  recursive  equations,  developed  in  [5],  for 
gain  computation. 

The  comparison  of  filters  [3]  and  [5]  against  the 
optimum  estimator  is  presented  in  table  3.1  and  figure  3.3. 
In  these  results  the  coefficient  of  correlation  is  0.9, 
in  both  directions,  and  the  variance  of  X  is  1.0.   The 
variance  of  noise  is  in  the  range  0.01-1.0,  therefore  the 
signal-to-noise  ratio  goes  from  0-20  dB. 

From  figure  3.3  and  table  3.1,  we  see  that  filter  [3] 
is  better  than  [5] ,  but  both  are  quite  close  to  the  optimum 
filter.   In  the  worst  case,  the  error  variance  of  [3]  is 
6.5%  greater  than  the  optimum,  and  [5]  is  15%.   Also,  the 
error  variance  of  [5]  is  8%  greater  than  that  of  filter 
[3].   It  was  observed,  from  other  results,  that  [3]  and 
[5]  approach  the  optimum  for  lower  coefficients  of  corre- 
lation and/or  low  noise. 

Another  experiment  was  performed  to  change  the  gain  to  see 
what  MMSE  can  be  achieved  with  such  a  filter  structure. 
Figure  3.4  and  table  3.2  show  that  filter  [3]  is  quite  close 
to  the  minimum  error  variance,  although  it  is  not  exactly 
the  minimum  as  can  be  verified  in  figure  3.5  and  table  3.3. 
The  conclusion  is  that  filter  [3]  is  better  than  [5] ,  but 
it  is  not  the  best  that  can  be  done.   As  a  matter  of  fact 
the  present  method  of  comparison  can  always  be  used  to 
compute  the  best  gain,  although  it  is  computationally  complex. 
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The  reason  why  filter  [3]  is  better  than  [5]  is  that, 

although  it  assumes  ( incorrectly)  that  the  estimation  error 

is  uncorrelated  with  the  measurements  in  region  Y   ,    , 

m-l,n-l 

the  gain  is  chosen  such  that  the  error  is  uncorrelated  with 
y(m,n).   On  the  other  hand,  the  error  of  filter  [5]  is  not 
uncorrelated  with  y(m,n),  although  it  minimizes  the 
up-dated  error  variance  at  each  point.   What  happens  is  that 
the  value  of  such  minimum  is  not  only  dependent  on  the 
previous  error  variances,  but  also  on  the  covariances  of 
the  "past"  errors,  since  they  are  correlated. 

The  important  conclusion   is  that   the 
recursive  filters  [3],  [5]  and  the  one  of  section  C,  although 
not  optimum,  are  quite  close  to  optimality.   Filter  [3]  is 
the  best  and  simple  enough  to  be  extended  to  three  dimensions, 
in  order  to  exploit  recursively  the  correlation  in  time. 
This  will  be  done  in  the  next  chapter. 
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Table   3.1 


Noise 

Optimum 

Filter  [3] 

Filter  [5] 

Variance 

0.01 

0.0083729        0.0084204 

0.0083936 

0.04 

0.0262313        0.0269059 

0.0266656 

0.09 

0.0477314        0.0497128 

0.0494879 

0.16 

0.0706634        0.0743120 

0.0747814 

0.25 

0.0940204        0.0994702 

0.1014409 

0.36 

0.1172805        0.1245305 

0.1287582 

0.49 

0.1401542        0.1491169 

0.1562232 

0.64 

0.1624769        0.1730206 

0.1834437 

0.81 

0.1841581        0.1961401 

0.2101098 

1.00 

0.2051533        0.2184254 

0.2359806 

Picture  Variance  =  1.0 

Picture  Correlation  =  0.9 
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PICTURE  VARIANCE  =1.0 
PICTURE  CORRELATION  =  0 
(o)  -  optimum  filter 
(*)  -  filter  [3] 
(+)  -  filter  [5] 
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Figure  3.3 
Comparison  of  the  Filters 
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Table    3.2 


Gain 

Error  Variance 

0.07982749               0.26742077 

0.09579301               0.24629241 

0.11175853               0.23272502 

0.12772405               0.22441578 

0.14368957               0.21995461 

0.15965503               0.21842539 

Filter  [3] 

0.17562050               0.21918494 

0.19158608               0.22175616 

0.20755148               0.22577757 

0.22351706               0.23097461 

0.23948258               0.23713821 

Filter  [5] 

Picture  Variance  =  1.0 

Noise  Variance  =  1.0 

Picture  Correlation  =  0.9 
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PICTURE  VARIANCE  =1.0 
NOISE  VARIANCE  =1.0 
PICTURE  CORRELATION  =0.9 
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Figure  3.4 
Error  Variance  vs  Steady-State  Gain 
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Table  3.3 


Gain 


Error  Variance 


0.29999995 

0.30999994 

0.31999993 

0.32999992 

0.33999991 

0.34999990 

0.35999990 

0.36999995 

0.37999994   . 

0.38999993 

0.39999992 


0.09970987 
0.09923345 
0.09892100 
0.09875959 
0.09873748 
0.09884429 
0.09907085 
0.09940886 
0.09985095 
0.10039049 
0.10102153 


Filter  [3: 


Best  Gain 


Filter  [5] 


Picture  Variance  =  1.00 
Noise  Variance  =  0.25 
Picture  Correlation  =  0.90 
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PICTURE  VARIANCE  =1.00 
NOISE  VARIANCE  =  0.25 
PICTURE  CORRELATION  =0.90 
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Figure  3.5 
Error  Variance  vs  Steady-State  Gain 
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IV.   THREE-DIMENSIONAL  RECURSIVE  FILTER 

A.  INTRODUCTION 

The  two-dimensional  recursive  filters,  presented  in 
Chapter  III,  use  only  the  spatial  correlation  between 
pixels  in  order  to  estimate  the  gray  levels  of  noisy  pic- 
tures.  In  this  chapter  we  introduce  a  recursive  filter 
that  takes  advantage  of  both  correlations  in  space  and 
in  time,  when  one  has  a  group  of  pictures  sequenced 
in  time.   Experimental  evidence,  presented  in  [1],  shows 
that  television  pictures  are  correlated  in  time.   This  is 
also  an  intuitive  result,  since  knowing  one  frame  we  often 
can  guess  the  next. 

The  filter  developed  here  is  an  extension  of  the  two- 
dimensional  recursive  filter  [3],  since  it  is  the  best, 
according  to  the  results  of  Chapter  III.   In  any  case,  its 
performance  will  be  analyzed,  in  order  to  evalute  the 
improvement  resultant  in  using  the  time  correlation. 

B.  FILTER  DESIGN 

The  filter  will  be  developed  under  the  following 
conditions : 

(a)   The  time-frame  is  modeled  as  an  homogeneous  random 
field  with  zero-mean  (or  known  mean)  and  autocorrelation 
function  R(i,j,k),  where  i,j  and  k  are  the  distances  between 
pixels  in  the  vertical,  horizontal  and  time  coordinates, 
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respectively.   The  autocorrelation  is  like  the  one  intro- 
duced in  Chapter  II,  with  kernels  given  by  equation 
(2.26)  . 

(b)   The  dynamic  model  is  given  by  the  partial  difference 
equation: 

JX(m,n,t)   =  A  X  +  W(m,n,t)  (4.1) 

where  A  is  a  row  vector  of  coefficients  and  X  is  a  column 
vector  of  adjacents.   The  modeling  error  W(m,n,t)  is  white 

noise  and  uncorrelated  with  the  X's  in  region  X     (see 

3     mnt 

figure  4.1). 

In  order  to  make  clear  the  present  derivation,  let's 
assume  a  specific  form  for  the  autocorrelation,  say,  first 
order  in  all  dimensions.   The  development  remains  valid  for 
the  k-order  kernels  given  by  equation  (2.26). 

The  autocorrelation  is: 


R(i,j,k)   =  a2  pv'1l  phljl  pt'kl...   i,j,k  =  0,±1,±2, 

(4.2! 

The  coefficients  are: 


A   =   [p    p,   p,   -p  p,   -p  p.   ~Pt,P.   P  Pt.Pj.1       (4.3 
—       v   h   t    vKh    vKt    h  t    v  h  t 


The  adjacents  (see  figure  4.2)  are: 
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time 


Figure  4.1 

Definition  of  the  region  X     .  :   the  region 

m,n ,  t 

includes  the  past  frames  and  the  pixels  shown 


m-1- 
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l-l  n 


time  =  t-1 
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Figure  4 . 2 
Configuration  of  the  adjacents 
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XT   =   [X(m-l,n,t)   X(m,n-l,t)   X(m,n,t-1)   X(m-l,n-l,t) 


X(m-l,n,t-l)   X(m,n-l,t-l)   X(m-1 ,n-l , t-1) ] 

(4.4) 

The  variance  of  W  is 

E[W2(m,n,t)]   =   Q  =   a2 (l-pv2) (l-ph2) (l-pt2) 

(4.5) 

(c)   The  pictures  are  contaminated  with  zero-mean  white 
noise,  uncorrelated  with  the  X's. 

y(m,n,t)   =   X(m,n,t)  +  v(m,n,t)  (4.6) 

The  noise  variance  is: 


E(v2(m,n,t))   =   r 


1.   Filter  Structure 

Since  we  want  to  generate  a  random  field  X  as  close 
as  possible  to  X,  let's  choose  a  structure  similar  to  the 
dynamic  model  (4.1): 

X(m,n,t)   =   B  X  +  K(m,n, t) y (m,n, t)  (4.7) 

In  what  follows  we  will  drop  the  argument  (m,n,t) , 
where  it  could  not  cause  confusion. 
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Observe  that  X  can  be  written  as  a  linear  combina- 
tion of  all  measurements  within  region  Y   , ,  defined  in 

s     mnt 

figure  4.3.   The  optimum  recursive  filter  must  have  the 
gains  such  that  the  weights  of  the  measurements  are  exactly 
the  same  as  for  the  non-recursive  estimator,  constrained 
to  the  same  data  set  Y    .   The  criteria  of  optimality  is 
minimization  of  the  Mean  Square  Error  (MSE) .   It  will  be 
seen  that  it  is  impossible  to  have  an  optimum  recursive 
filter,  constrained  to  the  dynamic  model  (4.1). 
2 .   Orthogonality  Principle 

A  necessary  condition  of  optimality  is  that  the 
estimation  error  be  uncorrelated  with  the  data  set.   Let's 
apply  this  condition  to  find  the  coefficients  B  and  K. 
The  estimation  error  is : 

e(m,n,t)   =   X  -  X  (4.8) 

The  orthogonality  condition  is : 

E[e(m,n,t)y(p,q,r) ]   =   0  (4.9) 


for  all  (p,q,r)  in  Y     (see  figure  4.3) 

mnt 

First,  let's  apply  (4.9)  to  the  measurement  y(m,n,t) 


E(ey)   =   E[(X-X)y]   =   E [y (X-BX-Ky) ] 


=      E[Xy-Ky2-ByX] 
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time 


Figure  4.3 
Definition  of  the  region  Ym   t 
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Using  equations  (4.2)  and  (4.6) 


2 
E(ey)   =   (l-K)o  -  Kr  -  BE (XX)   -   0 


K(a2+r)  +  BE(XX)   =   a2  (4.10) 


Let's  apply  (4.9)  to  the  rest  of  the  measurements 
Substituting  (4.7)  into  (4.8): 

e      =      X-X   =   X-Ky-BX 
(l-K)X  -Kv-BX  +  Be 


e   =   (1-K)  [X  -  ^V  B  xl  "  Kv  +  B  e      (4.11) 


Substituting  (4.11)  into  (4.9): 

BE  [y(p,q,r)e]  +  (1-K) E [ (X  -  j±-  B  X) y (p,q,r ) ]   =   0 

(4.12) 
for  y(p,q,r)  ^  y(m,n,t),  and  in  Ymnt- 

The  problem  now  is  to  choose  the  coefficients  B 
and  K,  such  that  equations  (4.10)  and  (4.12)  are  satisfied. 
At  this  point  we  follow  [3]  by  choosing 

B   =   (l-K)A  (4.13) 
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With  such  a  choice,  the  second  term  of  (4.12) 
becomes : 

(l-K)E[W(m,n,t)y(p,q,r)  ] 

Where  we  have  used  (4.1),  and  this  term  vanishes 

because  W(m,n,t)  is  uncorrelated  with  all  measurements  in 

region  X   ,.   Substituting  (4.13)  into  (4.10): 
3     mnt  ^ 

a2  +  b 
K(m,n,t)   =  —= 2 (4.14) 

a   +  b   +  r 

P 

where  we  have  defined: 


X  (m,n,t)   =  A  X 
p  —  — 


b  (m,n,t)   -   E(XX  ) 


(4.15) 


Equation  (4.15)  seems  to  be  an  adequate  choice  for 
the  first-step  predictor,  also  (4.14)  is  an  intuitive  result 
for  the  gain,  since  it  is  unity  for  zero  noise  (r=0)  and 
decreases  when  noise  increases. 

Unfortunately,  the  first  term  of  (4.12)  remains  and 
we  could  not  force  it  to  vanish.   Choosing  K  by  (4.14)  we 
have  forced  the  estimation  error  to  be  uncorrelated  with 
y(m,n,t),  but  it  is  not  correct  to  conclude,  by  induction, 
that  the  estimation  errors  of  the  adjacents  (vector  e)  are 
uncorrelated  with  the  other  measurements . 
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Take,  for  example,  the  error: 

e(m,n,t-l)   =   X(m,n,t-1)  -  X(m,n,t-1) 

The  estimate  didn't  use  any  measurements  of  frame 

t,  therefore  the  error  can  not  be  uncorrelated  with  them. 

As  a  matter  of  fact,  the  errors  might  only  be 

uncorrelated  with  the  measurements  within  region  Y   ,    n  L    , , 

3     m-l,n-l,t-l 

In  reference  [3],  for  the  two-dimensional  case,  it  is 
stated  that  the  error  is  uncorrelated  with  all  measurements 
in  a  similar  region.   Unfortunately,  that  is  not  correct. 
The  estimation  error,  in  the  two  or  three  dimensional  cases, 
is  uncorrelated  only  with  y(m,n,t),  because  K  was  computed 
under  such  conditions,  and  with  the  measurement  y( 1,1,1). 
This  can  be  seen  from  (4.12)  and  (4.13): 

E[ey(p,q,r)]   =   (1-K) E [A  e  y (p,q,r) ]         (4.16) 

for   (p,q,r)  ^  (m,n,t) 

Initializing  the  filter  with  the  mean  value  of  X 
(=0),  it  can  be  verified,  using  (4.16),  that  e(m,n,t)  is 
uncorrelated  with  y( 1,1,1). 

Although  equation  (4.14)  is  not  the  optimum  choice 
for  the  gain,  we  have  shown  in  Chapter  III  that  it  was  the 
best  in  the  two-dimensional  case,  therefore  we  hope  it  is 
also  good  in  three  dimensions.   This  will  be  verified  at 
the  end  of  this  chapter. 
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Using  (4.7)  and  (4.13),  the  filter  equation  becomes: 


X(m,n,t)   -   (l-K)A  X  +  Ky  (4.17) 


Alternatively,  using  (4.15) 


X(m,n,t)   =   (l-K)X   +  Ky  (4.18) 


3.   Gain  Computation 

Let's  develop  a  recursive  equation  to  compute  b 
and,  therefore,  the  gain.   Define  a  function: 


F    (m,n,t)   =   E[X(m,n,t)X(p,q,r)] 


Using  (4.17)  : 


F     =  [1-K(p,q,r) ]A  F  +  K(p,q,r)R(m-p,n-q,t-r)       (4.19) 


where: 


T 
F   =  [F  F  F  F  F 

-      p-l,q,r    p,q-l,r    p,q,r-l    p-l,q-l,r    p-l,q,r-l 


F     .    .   F  1  .  .]  (4.20 

p,q-l,r-l    p-l,q-l,r-lJ 


Initializing  with  X  equal  to  the  mean  of  X,  outside 
the  picture,  the  initial  conditions  for  F  are: 
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F      (m,n,t)   =   0    for    p,q  or  r  =  0 
p,q,rv  ^'H 


Now,  using  (4.15)  and  (4.19) 


b  (m,n,t)   =   A  F  (4.21) 

P  p 


where  F   is  the  value  of  F  for  (p,q,r)  =  (m,n,t) . 
R 

Summary  of  gain  calculation: 
(a)   Use  (4.19)  to  compute  each  component  of  the  vector 


F  . 
-P 


(b)   Use  (4.21)  to  compute  b  (m,n,t) 


(c)   Compute  K(m,n,t)  by  (4.14) 

Observe  that  the  calculation  of  each  gain  K(m,n,t) 
requires  scanning  the  time-frame  from  (1,1,1)  through  the 
pixel  (m,n,t).   Fortunately,  as  will  be  seen,  after  a  few 
frames  the  gain  reaches  steady  state,  therefore  the  compu- 
tation can  be  reduced  to  the  top  left  corner  of  a  few 
frames. 

C.   PERFORMANCE  OF  THE  FILTER 

The  method  used  in  Chapter  III,  for  the  two  dimensional 
case,  can  be  easily  extended  to  three  dimensions.   The 
variance  of  the  estimation  error  is : 


P(m,n,t)   =   E[(X  -  X)2] 


=   E(X2)  +  E(X2)  -  2E(XX) 
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or: 


P(m,n,t)   =  a2   +   o    2   -  2b  (4.22 


where : 


2        2 

E(XZ)   =   a 


~2         2 

E(XZ)   =  a^ 


E(XX)   =   b 


The  covariance  b  can  be  calculated  using  b   and  K: 

P 


b(m,n,t)   =   E(XX)   =   (l-K)E(XX  )  +  KE(Xy) 


b(m,n,t)   =   (l-K)b   +  Ka2  (4.23) 


To  compute  the  variance  of  the  estimator,  write  X  as 

a  linear  combination  of  the  measurements  in  region  Y   , : 

mnt 


X(m,n,t)   =   cTY  (4.24! 


where  _c  is  a  column  vector  of  coefficients ,  function  of 
the  gains,  and  Y  is  a  column  vector  containing  all  measure- 
ments in  Y 

mnt 

The  variance  of  X  is: 
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2       T 
a/      =      c\c 


(4.25) 


where  R   is  the  correlation  matrix  of  the  vector  Y: 


Ry  =   E(YY' 


The  key,  in  this  computation,  is  the  method  of  finding 
the  coefficients  c.   This  can  be  easily  accomplished 
using  the  filter  equation  (4.17)  with: 


y(i,j,k)   =   < 


(i,j,k)  =  (p,q,r) 


(i,j,k)  7*  (p,q,r) 


X(i,j,k)   =   0   for   k  =  (r-1)  or 

i  =  (p-1)  or 
j  =  (q-D 

Initializing  (4.17)  as  above,  and  starting  at  (p,q,r), 
the  resultant  value  at  (m,n,t)  will  be  the  coefficient  of 
y(p,q,r)  in  the  estimate  X(m,n,t).   In  this  way  all  coeffi- 
cients can  be  found  and,  therefore,  the  error  variance. 

We  are  also  interested  in  the  first-step  predictor 
given  by  (4.15).   Its  error  variance  is: 


P  (m,n,t)   =   E[(X  -  X  )*] 
P  P 


(4.26) 
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It  can  be  related  with  P(m,n,t)  using  (4.18)  and  (4.26) 


X  -  X  =   X  -  (l-K)X   -  Ky 

P 


(1-K) (X  -  X  )  -  Kv 


Squaring  and  taking  expectations 


P   -   (1-K)2P   +  K2r 
P 


P  (m,n,t)   =   ^— ~  (P  -  K2r)  (4.27) 

P  (1-K)Z 


D.   RESULTS 

In  figures  (4.4)  through  (4.9)  some  results  of  gain 
calculations  are  shown.  For  the  first  frame  the  gain  is 
the  same  as  in  [3]  for  two  dimensions.   From  these  results 
we  can  observe  that  the  gain  reaches  a  steady-state  value 
very  fast,  at  about  frame  number  4.   Also,  for  the  same 
frame,  the  steady-state  is  reached  at  about  pixel  (4,4)  . 
That  is  an  interesting  result,  since  it  simplifies  substan- 
tially gain  calculations.   Also  we  can  think  of  using  a 
constant  gain  for  the  whole  time-frame,  or  a  few  gains 
for  the  first  4  frames. 

In  tables  4.1  through  4.3  a  comparison  is  shown 
between  the  three-dimensional  filter,  estimate  and  prediction, 
and  the  two-dimensional  recursive  filter.   It  can  be  seen 
in  table  4.1  that  exploitation  of  time  correlation  can  be 
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quite  advantageous  in  cases  where  time  correlation  is 
very  high,  compared  with  spatial  correlations.   In  this 
example,  the  estimation  error  variance  is  reduced  by  about 
33%,  due  to  the  use  of  time  correlation.   In  table  4.2 
a  comparison  with  equal  amounts  of  correlation  in  time  and 
space  is  shown.   In  this  example  the  improvement  was  about 
10%.   It  can  be  seen  in  table  4.3  that  exploitation  of  time 
correlation  is  not  so  advantageous  when  the  images  are 
more  correlated  in  space  than  in  time.   In  this  example 
the  improvement  was  only  around  6%. 
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Figure  4.4 
Gains  for  the  First  Frame 
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Figure  4 . 5 
Gains  for  the  Time-Frame 
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Figure  4 . 6 
Gains  for  the  First  Frame 
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Figure  4.7 
Gains  for  the  Time-Frame 
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Gains  for  the  First  Frame 
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Gains  for  the  Time-Frame 
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Table    4.1 


Error 

Variance 

Noise 
Variance 

Three- Dimensions 

Two- Dimensions 

Estimation 

Prediction 

Estimation 

Prediction 

1.0 

.236          .293 

.357 

.554 

0.8 

.210          .267 

.318 

.527 

0.6 

.178          .236 

.271 

.493 

0.4 

.140          .197 

.212 

.450 

0.2 

.089          .144 

.132 

.387 

Picture  Variance  =  1.00 

Spatial  Correlation  =0.70 

Time  Correlation  =  0.95 

• 

- 

Table    4.2 


Error 

Variance 

Noise 
Variance 

Three-Dimensions 

Two-Dimensions 

Estimation   Prediction 

Estimation 

Prediction 

1.0 

.269          .351 

.301 

.427 

0.8 

.240          .324 

.268 

.400 

0.6 

.206          .293 

.229 

.367 

0.4 

.162          .251 

.180 

.324 

0.2 

.103          .191 

.114 

.263 

Picture  Variance  =  1.0 

Spatial  Correlation  =  0.8 

Time  Correlation  =  0.8 
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Table    4.3 


Error  Variance 

Noise 
Variance 

Three-Dimensions 

Two-Dimensions 

Estimation 

Prediction 

Estimation 

Prediction 

1.0 

.280          .376 

.301 

.427 

0.8 

.250          .349 

.268 

.400 

0.6 

.214          .317 

.229 

.367 

0.4 

.168          .274 

.180 

.324 

0.2 

.107          .213 

.114 

.263 

Picture  Variance  -    1.0 

Spatial  Correlation  =  0.8 

Time  Correlation  =  0.7 
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V.   HYBRID  FILTERS 

A.   INTRODUCTION 

In  this  chapter  a  new  class  of  image  filters  is  intro- 
duced, called  hybrid  filters.   The  recursive  filters, 
presented  in  Chapters  III  and  IV,  utilize  only  part  of  the 
data  set,  arbitrarily  defined  as  "past"  measurements.  Since 
the  whole  picture  is  often  available,  an  optimal  smoother 

might  be  sought  in  order  to  utilize  all  the  information 
available.   For  the  two-dimensional  recursive  filters  of 
Chapter  III,  the  smoother  would  be  the  optimum  combination 
of  the  four  estimates  obtained  by  scanning  the  picture 
starting  at  each  corner.   The  first  difficulty  is  that 
those  filters  are  not  optimum  resulting  in  a  sub-optimum 
smoother.   Second,  the  smoother  would  require  scanning  the 
picture  four  times,  therefore  increasing  substantially  its 
complexity  of  implementation. 

The  hybrid  filter  introduced  here  is  a  smoother  that 
combines  optimally  the  estimate  of  the  recursive  filter 
(two  or  three  dimensions)  with  an  arbitrary  set  of  "future" 
measurements.   It  will  be  applied  in  picture  enhancement 
and  compared  against  the  recursive  filter  as  well  as  the 
non-recursive  filter  presented  in  [6].   Some  examples  of 
hybrid  filters  designed  to  predict  the  pixel  gray  level  will 
also  be  presented.   These  filters  will  be  applied  in 
Chapter  VI  for  purposes  of  target  detection. 
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The  hybrid  filter  is  particularly  useful  in  some 
applications  where  the  pictures  are  not  from  a  fixed 
scenario.   For  example,  the  camera  has  a  movement  in  order 
to  follow  some  target.   In  such  cases,  the  background  may 
change  so  much  that  past  frames  don't  carry  enough  informa- 
tion about  the  present  frame.   In  this  case,  the  three- 
dimensional  recursive  filter  can't  be  applied,  but  a  hybrid 
filter  can  be  designed  using  recursively  the  measurements 
in  the  present  frame,  and,  non-recursively ,  the  "crude" 
measurements  in  the  previous  frame. 

B.   TWO-DIMENSIONAL  HYBRID  FILTERS 

In  this  section  we  present  two  examples  of  hybrid 
filters  in  two-dimensions.   One  is  a  smoother  designed  to 
enhance  pictures  contaminated  with  additive  white  noise. 
The  other  is  a  filter  designed  to  predict  the  pixel  gray 
level,  based  on  noisy  measurements  of  previously  scanned 
pixels.   Here  we  assume  the  picture  is  scanned  row  by  row 
from  top  to  bottom. 

1.   Two-Dimensional  Hybrid  Smoother 

The  recursive  filters  presented  in  Chapter  III 

utilize  the  measurements  in  region  Y    (see  figure  3.2) 

J     mn 

to  have  an  estimate  X(m,n) .   To  improve  this  estimate  we 
are  going  to  use  also  a  selected  set  of  measurements  in 
the  neighborhood  of  (m,n) ,  say  Y,  and  combine  optimally 
with  X(m,n) ,  in  order  to  have  a  better  estimate  X, (m,n) . 
Let's  choose  for  Y  the  set  of  five  neighbors  shown  in 
figure  5.1. 
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Figure  5.1 
Two-dimensional  hybrid  smoother 
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Figure  5.2 
Non-recursive  filter 
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Thus 


X,  (m,n)   =   h  2, 


!5.1) 


where 


2.  = 


X(m,n) 

y  (m+l,n-i; 
y (m+l,n) 
y  (m+1  ,n+l! 
y (m,n+l) 
y  (m-l,n+i; 


h   ■ 


hr 


Let's  calculate  the  weights  h  in  order  to  minimize  the 
mean-square-error  in  the  estimate.   The  error  is 

e(m,n)   =   X  -  X.   =   X  -  hTg_ 

The  error  variance  is 


P(m,n)   =   E[(X  -  hTg_)2] 


(5.2) 


Differentiating  (5.2)  with  respect  to  h  and  equating 
to  zero 


h  =   R  _1  R^ 
gg    ^c 


(5.3) 
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where 


Rgg   =   E(22T) 


RXg   =   E(X2) 


The  measurement  equation  is 

y(m,n)   =   X(m,n)  +  v(m,n)  (5.4) 

where  v  is  zero-mean  white  noise,  uncorrelated  with  X, 
having  variance 


2 

E [v  (m,n) ]   =   r 


The  gray  level  X  is  a  zero-mean  (or  known  mean) 
homogeneous  random  field  with  autocorrelation 

R(i,j)   =   E[X(k,£)X(k+i,£+j) ]  (5.5! 


The  elements  of  the  autocorrelation  matrix  R   ,  that 

gg 

don't  include  X(m,n),  are  easily  found  using  equations 
(5.4)  and  (5.5).   To  find  the  elements  which  include  x, 
observe  that  in  Chapter  III  a  method  was  developed  to  cal- 
culate error  variance  of  recursive  filters  [3,5],  where 
E(X  )  is  also  computed  as  well  as  E(XX) .   It  can  be 
verified  that  the  same  method  can  be  used  for  higher  order 
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recursive  filters,  though  in  this  derivation  we  restrict 
ourselves  to  recursive  filter   [3]. 

Therefore,  using  (3.11)  and  (3.13): 

~2         2 
E(XZ)   =   aez 

(5.6; 

E(XX)   =   b 

It  can  be  verified  that  the  covariances  between 
X  and  the  measurements  Y  can  be  expressed  as  a  function 
of  b  given  by 


E[Xy(m+l,n-l) ]   =   pvphb 


E[Xy(m+l,n)]   =   pvb 


E[Xy(m+l,n+l) ]   -   PyPhb  (5.7; 


E[Xy(m,n+l)]   =   phb 


E[Xy(m-l,n-l)  ]   ■   pvPhb 


Therefore  we  have 
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gg 


a  p    p,b      p   b        p    p,b      p,  b 

e         'v'n  v  KvHh        Hh 


2  2    2 


pvphb 


2    2  2       2    2 


a   +r        p.  a        p.     a        p    p,     a        p      p,     a 
Kh  h  ^vMh  v    Kh 


2  2  2  2         2 

a   +r      p,  a  p,  p    a  p      p,  a 

*n  KhMv  v     h 


2^  2  2    2 

a   +r        pva  pv  a 


2  2 

a   +r  pa 

v 


2^ 

a   +r 


Xg 


vMh 


P    a 
v 


P      P-ua 


pha 


P    Pt~a 
Kv  h 


where  we  have  made  use  of 


o  i  ■     ■  \       2    I  i      j 
R(i,])   =   a   Pv'  '  Ph' 


(5.8 
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a.   Performance  and  Comparison 

The  error  variance  of  the  hybrid  smoother  is 
calculated  substituting  (5.3)  into  (5.2): 


P(m,n)   =   a2  -  hTR  (5.9) 

Ay 


At  this  point  it  is  interesting  to  compare  the 
hybrid  smoother  and  the  recursive  filter  [3],  since  we 
know  how  to  calculate  the  error  variance  for  both.   It  is 
also  interesting  to  include  in  such  comparison  the  non- 
recursive  filter  presented  in  [6] .   This  filter  uses  a 
small  window  of  3x3,  shown  in  figure  5.2,  to  estimate  the 
pixel  gray  level  at  the  center,  minimizing  the  mean-square- 
error.   The  hybrid  smoother  must  have  superior  performance, 
since  it  utilizes  the  same  measurements  of  [3]  and  [6] 
together.   However,  we  can't  say  which  is  better  whether 
[3]  or  [6],  since  they  use  distinct  data  sets.   Although 
filter  [3]  uses  a  larger  data  set  than  [6],  the  former 
uses  all  the  closest  adjacents  to  the  estimated  pixel 
(m,n)  . 

In  table  5 . 1  some  numerical  results  are  shown 

to  help  the  comparison.   As  expected,  the  hybrid  filter 

presented  the  best  performance  with  error  variance  about 

25%  smaller  (for  p  =  0.94)  than  filter  [6].   The  non-recursive 

filter  [6]  presented  better  performance  than  the  recursive 

filter  [3].   The  error  variance  of  the  optimum  interpolator 

constrained  to  the  observations  in  Y    is  also  included  in 

m,n 
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Table  5.1 


Error  Variance 

Correlation 

Hybrid 

Filter  [6] 

Filter  [3] 

Optimum 
Interpolator 

0.60 
0.70 
0.80 
0.90 
0.92 
0.94 
0.96 

0.334 
0.279 
0.217 
0.145 
0.128 
0.112 
0.111 

0.346 
0.298 

0.245 
0.182 
0.167 
0.152 
0.136 

0.400 
0.357 
0.300 
0.218 
0.197 
0.173 
0.146 

0.399 
0.355 
0.294 
0.205 
0.181 
0.155 
0.124 

Picture  Variance  =  1.0 
Noise  Variance  =  1.0 

this  table.   It  is  interesting  to  see  that  the  use  of  just 
5  non-causal  observations  by  the  hybrid  filter  were  enough 
to  fully  compensate  the  suboptimality  of  the  recursive 
filter  [3] .   Observe  that  the  9-point  non-recursive  filter 

[6]  also  presented  better  performance  than  the  optimum 
interpolator,  even  for  spatial  correlation  as  high  as  0.94. 
This  indicates  that  most  of  the  information  about  X(m,n) 
resides  in  its  nearest  neighbors.   This  motivates  the  deriva- 
tion of  simple  filters  like  the  hybrid  filter  introduced 
here.   It  is  much  simpler  than  the  optimum  recursive  filter 

[9]  with  insignificant  loss  in  performance.   It  also  has 
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the  same  order  of  complexity  as  the  non-recursive  filter 
[6],  but  superior  performance.   Complexity  here  means  the 
number  of  multiplications,  which  is  9  for  the  non-recursive 
filter  and  10  for  the  hybrid  filter. 

The  hybrid  filter  may  be  seen  as  a  way  of 
artificially  increasing  the  window  of  filter  [6] . 
b.   Experimental  Results 

In  this  section  we  present  the  results  of  some 
experiments  where  the  filters  were  used  in  picture  enhance- 
ment.  In  this  experiment,  the  picture  of  figure  5.3  was 
contaminated  with  white  noise.   Assuming  knowledge  of  the 
noise  variance,  the  picture  was  modeled  by  the  method 
presented  in  Chapter  II.   With  this  model  and  noise  vari- 
ance, we  designed  the  three  filters:   hybrid,  recursive 
and  non-recursive.   The  theoretical  performance  was  computed 
for  each  filter,  as  in  the  last  item.   A  signal-to-noise 
ratio  (S/N)  was  defined  as  the  ratio  between  the  variance 
of  the  original  picture  (figure  5.3)  and  the  variance  of 
the  noise.   A  processing  gain  was  defined  as  the  ratio  of 
(S/N)'s  at  the  filters  output  and  input: 


*      (S/N)i 


Table  5.2  summarizes  the  results.   It  can  be 
seen  that  the  computed  performance  is  pretty  close  to  the 
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Figure  5.3 
Original  image  of  a  power  plant 
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results  of  the  experiments.   The  hybrid  is  clearly  superior 
to  the  others  and  the  non-recursive  filter  is  better  than 
the  recursive,  except  for  the  very  low  S/N  of  -3  dB,  when 
the  former  was  a  little  better.   However  the  subjective 
evaluation  of  the  pictures  (figures  5.  4  through  5.15)  is 
more  difficult.   In  our  evaluation  the  hybrid  filter  had 
superior  performance,  therefore  validating  the  theoretical 
results,  as  well  as  the  measurements  of  table  5.2.   However, 
the  results  of  the  non-recursive  filter,  in  our  subjective 
evaluation,  were  inferior  to  the  recursive  filter  in  all 
three  experiments,  contradicting  the  measurements  of  table 
5.2.   Probably  the  mean-square-error  is  not  a  good  figure 
of  merit  for  visual  perception,  however  there  is  no  doubt 
that  considerable  improvement  was  achieved  by  processing 
the  noi.sy  pictures  using  such  criteria. 
2 .   Two-dimensional  Hybrid  Predictor 

Assuming  that  the  picture  is  scanned  row  by  row, 
we  may  define  "past"  measurements  as  those  pixels  already 
scanned.   Then  the  gray  level  of  pixel  (m,n)  may  be  pre- 
dicted using  "past"  measurements.   In  figure  5.16  the 
configuration  of  a  hybrid  filter  designed  to  predict  X(m,n) 
is  shown.   This  filter  combines  optimally  the  prediction 
X  (m,n)  of  the  recursive  filter  and  the  measurements  Y, 
shown  in  figure  5.16.   The  result  is  an  improved  prediction 
X(m,n)  that  utilizes  all  the  information  contained  in  the 

measurements  Z    and  Y. 
mn     — 
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Figure  5.4 
Image  of  figure  5.3  with  additive  noise  (SNR  =  3  dB) 
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Figure  5.5 

Image  of  figure  5.4  filtered  by  the  hybrid  filter 

(SNR=  11.5  dB) 
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Figure    5.6 

Image   of    figure   5.4    filtered  by   the   recursive    filter 

(SNR  =    9.3    dB) 
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Figure    5.7 

Image  of  figure  5.4  filtered  by  the  9-point   non-recursive  filter 

(SNR=  10.4  dB) 
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Figure  5 . 8 
Image  of  figure  5.3  with  additive  noise  (SNR  =  0  dB) 
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Figure  5.9 
Image  of  figure  5.3  filtered  by  the  hybrid  filter 

(SNR  =  9.8  dB) 
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Figure  5.10 
Image  of  figure  5.8  filtered  by  the  recursive  filter 

(SNR  =  7.8  dB) 
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Image   of    figure    5 


Figure  5.11 

filtered  by  the  9-point   non-recursive  filter 
(SNR  =  8.2  dB) 
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Figure    5.12 
Image   of   figure    5.3   with   additive   noise    (SNR  =    -3    dB] 
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Figure  5.13 

Image  of  figure  5.12  filtered  by  the  hybrid  filter 

(SNR  =  8.1  dB) 
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Figure    5.14 

Image  of  figure  5.12  filtered  by  the  recursive  filter 

( SNR  =  6.6  dB ) 
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Figure    5 . 15 

Image   of    figure    5.12    filtered   by    the    9-point      non-recursive    filter 

(SNR   =    5.8    dB) 
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Figure  5.16 
Two-dimensional  hybrid  predictor 
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Figure  5.17 
Three-Dimensional  Hybrid  Smoother 
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The  design  of  this  filter  is  similar  to  the  hybrid 
smoother  of  the  previous  item,  therefore,  we  present  below 
the  results. 

The  autocorrelation  function  used  is 


R(i,j)   =  a2  pj1'  pj3 


(5.10: 


where 


The  predictor  is 


X(m,n)   =   h  g 


g  = 


X  (m,n) 
P 

y (m-l,n+l) 
y (m-1 ,n+2) 
y (m-2,n+l) 
y (m-2 ,n+2) 


h  = 


1 — 

— 

hl 

h2 

h3 

h4 

\ 

The  weights  h  are  given  by 


h   =   R   _1  Rv 
gg     Xg 


Using  the  results  of  Chapter  III  we  can  compute 

all  the  elements  of  R    and  R^  .   Let's  call: 

gg       Xg 
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E[X  2(m,n)]   =   a  2 


E[XXp]   =   bp 


(5.11) 


It  can  be  verified  that  the  cross-correlation 
between  X   and  Y  can  be  expressed  as  a  function  of  b  , 

P  P 

as  follows: 


E[X  y(m-l,n+l)  ] 
P 


P  Pi_b 
vKh  p 


E[X  y(m-l,n+2)  ] 

P 


p  p,  b 
v*n   p 


(5.12; 


E[X  y(m-2,n+l)  ] 
P 


P   p,  b 
v  Hh   p 


E[X  y  (m-2,n+2)  ] 


2   2, 

P   p,  b 
v  h   p 


Using  equations  (5.10),  (5.11)  and  (5.12),  we  can 
calculate  the  autocorrelation  matrix  R   and  the  correlation 

gg 

vector  R   : 
Xg 


R 


gg 


2  2       2         2   2 

a     p  p,b    p  p,  b    p   p,b    p   p,  b 
p    vKh  p   vHh      p   v  ^h  p   Mv  Hh  p 


a2+r 

2 

Pha 

2 

P  o 

P  Pua 

vKh 

a2+r 

2 

P  Pi^ 

v  h 

2 

P  o 

V 

2^ 

a  +r 

2 

p,  a 
■  h 

2, 

a  +r 

(5.13: 
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p  p,  a 
v  h 


2  2 

Xg        vKh 


2    2 

P  Q^o 
v  h 


2   2  2 
P   P-,  a 


C.   THREE-DIMENSIONAL  HYBRID  FILTERS 

In  this  section  we  present  two  hybrid  filters 
applied  to  pictures  sequenced  in  time.   The  first  filter 
is  a  smoother  to  be  used  in  image  enhancement  and  the 
other  is  a  predictor.   The  design  of  these  filters  is 
quite  similar  to  those  of  the  previous  section >. 
1.   Three-dimensional  Hybrid  Smoother 

The  filter  structure  is  shown  in  figure  5.17. 
The  recursive  filter  is  that  developed  in  Chapter  IV.   The 
estimate  X(m,n,t)  of  the  recursive  filter  is  optimally 
combined  with  the  set  of  measurements  Y,  in  order  to  have 
a  better  estimate  X, (m,n,t).   The  design  is  quite  the  same 
as  in  the  two-dimensional  case,  thus  we  present  only  the 
results. 

The  autocorrelation  used  for  the  time-frame  is 


R(i,j,k) 


2    i 
-  -a      p  ' 
v 


J    k 
ph   pt 


(5.14) 
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where : 


The  estimate  is 


X(m,n,t)   =  h  2 


r 


g  - 


X(m,n,t) 
y (m+l,n-l,t) 
y (m+l,n,t) 
y (m+l,n+l,t) 
y (m,n+l,t) 
y (m-l,n+l, t) 
y (m+l,n-l,t-l) 
y (m+l,n, t-1) 
y (m+l,n+l,t-l) 
y (m,n+l, t-1) 
y (m-l,n+l,t-l) 


h   = 


"^ 


h, 


10 


11 


The  weights  h  are  given  by 


h  =   R  _1  R 
-  gg    Xg 


Using  the  same  notation  as  in  Chapter  IV  ,  we  have 


E[X2 (m,n,t) ]   -  o    2 


E[XX]   =   b 


(5.15) 
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As  before,  the  correlation  between  X  and  the  measure- 
ments Y  can  be  expressed  as  a  function  of  b  as  follows 


E[X(m,n,t)y(m+i,n+j,t+k)  ]   =   pj1'  Ph     pt    b    (5-16 


where 

(i,j,k)   =   (1,-1,0) , (1,0,0) , (1,1,0) , (0,lk0) , 

(-1,1,0) (1,-1,-1)  (1,0,-1) , (1,1,-1) , 
(0,1,-1) , (-1,1,-1) 


Using  equations  (5.14),  (5.15)  and  (5.16),  we  can 
calculate  the  autocorrelation  matrix  R   and  the  correlation 

gg 

vector   R      ,    then   the   weights    h. 

Xg  3    — 

2.   Three-dimensional  Hybrid  Predictor 

This  filter  combines  optimally  the  first-step 

prediction  X  of  the  two-dimensional  recursive  filter  [3] 
P 

and  the  set  of  measurements  Y  shown  in  figure  5.18.   Since 
its  design  is  quite  similar  to  the  two-dimensional  case, 
we  present  only  the  results . 

The  autocorrelation  function  is  given  in  (5.14) 
and  the  hybrid  prediction  is 


A  m 

X(m,n,t)   =   h  g 


where : 
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time 


Figure  5.18 
Three-dimensional  hybrid  predictor 
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g  = 


X  (m,n) 
y 

y (m-l,n-l, t-1) 
y (m-l,n, t-1) 
y (m-l,n+l,t-l) 
y (m,n-l,t-l) 
y (m,n,t-l) 
y (m+l,n+l,t-l) 
y (m+l,n-l,t-l) 
y (m+l,n, t-1) 
y (m+l,n+l,t-l) 


r 


h   = 


h. 


h. 


h. 


hr 


h, 


hr 


h 


10 


The  weights  h  are  given  by 


h  =   R  -1  Rv 
-     gg    Xg 


Similarly  to  the  two-dimensional  case: 


~  2  2 

E[X  4]       =      a    ' 
P         P 


E[XXp]   =   bp 


(5.i7: 


And  the  correlation  between  X^  and  Y  is  given  by 

P     - 


E[X  (m,n,t)y(m+i,n+j,t-l) ]   =   Pv|x|  Ph ' : '  Ptbp    (5.18) 


where 
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(i,j)   =   (-1,-1)  ,  (-1,0)  ,  (-1,1)  ,  (0,-1)  ,  (0,0)  ,  (0,1)  , 
(1,-1) , (1,0) , (1,1) 

The  autocorrelation  matrix  R   and  the  correlation 

y  y 

vector  Rv   can  be  calculated  using  equations  (5.14),  (5.17) 

Xg 

and  (5.18),  then  the  weights  h  are  computed. 

D.   COMPARISON  OF  THE  PREDICTORS 

The  predictors  developed  here  will  be  used  in  the  target 
detection  problem  addressed  in  Chapter  VI.   In  this  section 
we  present  some  numerical  results  in  order  to  evaluate  its 
performance. 

In  Tables  5.3  through  5.5  the  error  variance  of  the 
recursive  and  hybrid  predictors  for  several  situations  is 
shown.   From  these  results  we  can  see  that  the  exploitation 
of  the  time  correlation  can  be  quite  advantageous,  mainly 
for  the  case  of  high  correlation  in  time  (table  5.3),  where 
the  error  variance  of  the  three-dimensional  filters  is  around 
50%.  of  that  given  by  the  two-dimensional  filters.   We  also  can 
observe  in  the  three  tables  that  the  3D-hybrid  filter  is 
better  than  the  3D-recursive  filter  for  observation  noise 
variances  above  0.2,  although  the  former  exploit  only  the 
time  correlation  with  the  previous  frame.   This  result  rein- 
forces the  basic  idea  of  the  hybrid  filters,  that  is  the  use 
of  all  the  neighbors  of  the  pixel  (m,n) ,  including  those 
that  are  not  causal. 


143 


Table    5.3 


Noise 
Variance 

Error  Variance 

three-dimensional 

two-dimensional 

recursive 

hybrid 

recursive 

hybrid 

1.0 
0.8 

0.6 
0.4 
0.2 

.293        .282 
.267        .258 
.236        .229 
.197        .195 
.144        .150 

.554         .502 
.527         .476 
.493         .445 
.450         .405 
.387         .350 

Picture  variance  =  1.0 
Pv  =  Ph  =  0.7 
Pt  -  0.95 

Table    5.4 


Noise 
Variance 

Error  Variance 

three-dimensional 

two-dimensional 

recursive 

hybrid 

recursive 

hybrid 

1.0 
0.8 
0.6 
0.4 
0.2 

.351        .303 
.324        .283 
.293        .260 
.251        .231 
.191        .192 

.427         .364 
.400         .340 
.367         .312 
.324         .277 
.263         .227 

Picture  variance  =  1.0 
pv  =  ph  =  0.8 
Pt  =  0.8 
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Table    5.5 


Noise 
Variance 

Error  Variance 

three-dimensional 

two-dimensional 

recursive 

hybrid 

recursive 

hybrid 

1.0 
0.8 
0.6 
0.4 
0.2 

.376        .338 
.349        .317 
.317        .292 
.274        .260 
.213        .216 

.427         .364 
.400         .340 
.367         .312 
.324         .277 
.263         .227 

Picture  variance  =  1.0 
pv  =  ph  =  0.8 
Pt  =  0.7 
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VI   TARGET  DETECTION 

A.   INTRODUCTION 

In  Chapter  II  we  have  developed  models  for  images  and 
for  a  sequence  of  images  in  time.   Using  such  models  we 
have  constructed  two  and  three  dimensional  recursive  and 
hybrid  filters  to  smooth  out  noise  in  images.   In  this 
chapter  we  are  interested  in  the  detection  of  targets  using 
low  contrast  images.   This  problem  is  usually  referred  to 
as  background  suppression,  where  background  means  a  kind 
of  texture  that  predominates  in  the  picture.   In  this  con- 
text, the  word  target  means  the  information  that  we  are 
interested  in  extracting  from  the  pictures.   It  may  be  a 
ship,  an  airplane  or  some  kind  of  texture  statistically 
different  from  that  of  the  background. 

Both  textures,  target  and  background,  are  modeled  as 
in  Chapter  II,  but  it  is  assumed  that  less  "a  priori" 
knowledge  is  known  about  the  target.   The  basic  idea  in 
detection  is  to  use  the  prediction  of  the  image  filters 
previously  developed.   Knowing  this  prediction  and  the  actual 
observation  of  the  pixel  we  develop  here,  a  decision  rule 
to  decide  to  which  texture  the  pixel  belongs. 
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B.   TARGET  AND  BACKGROUND  REGIONS 

Figure  6.1  shows  two  distinct  regions  in  the  picture: 
background  and  target  regions.   In  the  visible  spectrum 
what  happens  is  a  replacement  process,  we  have  background 
or  target.   On  the  other  hand,  in  the  infrared  spectrum 
what  seems  to  happen  is  some  combination  of  the  radiations 
from  the  target  (e.g.,  a  plane)  and  the  background (e . g. , 
clouds).   To  include  these  situations,  let's  assume  that  the 
gray  level  in  the  target  region  is  given  by 

y(m,n)   =   ax(m,n)  +  T(m,n)  +  v(m,n)  (6.1) 

This  model  assumes  a  linear  combination  of  target  and 
background,  normalized  with  respect  to  the  target,  and  also 
consider  an  additive  white  observation  noise  v(m,n), 
uncorrelated  with  both  target  and  background  textures .   The 
background  texture  x(m,n)  is  modeled,  as  in  Chapter  II,  as 
a  homogeneous  random  field  with  known  mean  and  autocorre- 
lation function  and  it  is  considered  independent  of  the 
target  texture.   The  target  region  is  similarly  modeled, 
but  the  amount  of  "a  priori"  knowledge,  for  most  applica- 
tions, may  be  restricted  to  the  mean  and/or  variance. 

The  background  region  is  modeled  by 

y(m,n)   =   x(m,n)  +  v(m,n)  (6.2) 
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Figure  6.1 
Background  and  Target  Regions 


False  Alarm 

Figure  6.4 
Conditional  probability  density  function  (case  I) 
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The  term  ax(m,n)  in  (6.1)  takes  into  account  a  possible 
correlation  between  the  two  regions.   The  weight  a  is  a 
factor  to  be  adjusted  according  to  the  specific  nature  of 
the  picture.   In  the  visible  spectrum  it  seems  reasonable 
to  assume  a  replacement  process,  therefore  a  =  0.   In  the 
infrared  spectrum  it  may  assume  a  constant  value  at  the 
edges  of  the  target  and  zero  inside  this  region,  provided 
we  assume  that  the  sensor  has  rejected  the  background. 

C.   LIKELIHOOD  RATIO  TEST 

In  this  section  we  will  construct  a  likelihood  ratio  to 
test  the  two  hypotheses  against  a  threshold.   We  will  first 
consider  the  general  case  of  deciding  between  two  random 
fields,  target  and  background,  where  the  required  "a  priori" 
knowledge  are  the  mean  and  autocorrelation  function  of  the 
background  and  the  mean  and  variance  of  the  target  and  the 
measurement  noise.   After,  we  will  work  some  special  cases 
and  derive  its  R.O.C.  (Receiver  Operating  Characteristic). 
The  detector  developed  here  will  be  a  Newman-Pearson  detector 
[15],  where  the  threshold  is  obtained  from  the  R.O.C.  sub- 
jected to  the  constraint  that  the  probability  of  false 
alarm  is  less  than  some  desired  value. 

The  two  hypotheses  are : 

Hypothesis  H   -  The  measurement  y(m,n)  is  in  the  back- 
ground region: 


y  H   =  y   =  x  +  v 
1  i  o     Jo 
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Hypothesis  H,  -  The  measurement  y(m,n)  is  in  the  target 


region: 


y  I  H,   =   y,   =   ax  +  T  +  v 


The  decision  rule  will  be  applied  to  each  pixel  using 
the  observation  y(m,n)  and  the  prediction  of  the  pixel  gray 
level  x  (m,n).   Assume  that  the  picture  is  being  scanned 
row  by  row,  from  top  to  the  bottom.   For  the  sake  of  clarity, 
assume  that  the  past  pixels  were  correctly  identified  as 
belonging  to  the  background  region.   The  image  filters 
developed  before  (recursive  or  hybrid)  are  used  to  give  the 
prediction  x  (m,n) .   We  are  looking  for  a  decision  rule 
based  only  on  y(m,n)  and  x  (m,n) . 

ir 

Let's  define  the  likelihood  ratio: 


p(x _y|H, ) 

L(y,x  )   =   In  — J2 —  (6.3! 

P(xpy|Ho) 


where  p(x  y|'*)  is  the  joint  probability  density  function, 
conditioned  to  the  hypothesis  H   or  H, . 
Applying  Bayes  rule : 


p(x  yl'H.)   =  p(x  |'H.)p(y|x  h.  )   i  =  0,1 


Since  the  prediction  x   is  independent  of  the  hypothesis 


p(x  \R)       =   p(x  |Ho) 
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Substituting  these  results  into  (6.3): 


L(y,x  )   =   In  aP  (6.4: 


p 

(y 

w 

p 

(y 

x    H    ) 

P    ° 

At  this  point  we  will  make  the  assumption  that  the 
conditional  probabilities  density  functions  in  (6.4)  are 
Gaussian. 

We  can  write 


y  H   =   x   +  v  -  e 

J  i  o       p         p 


yj^   ■   axp  +  v  -  aep 


Since  x   is  known,  the  conditional  densities  are  func- 
P 

tions  of  (v-e  )  and  (v-ae  +T) ,  where  e   is  the  prediction 

tr  tr  hr 

error.   In  figure  (6.2)  the  histogram  of  (v-e  )  is  shown  for 
the  noisy  image  in  figure  (5.8)  ,  using  the  first-step 
predictor  of  the  two-dimensional  recursive  filter  [3]. 
This  result  shows  a  curve  quite  close  to  a  normal  distri- 
bution, and,  therefore,  validates  the  above  assumption  for 
p(y|x  H  )  and  also  for  p(y|x  H,  )  ,  provided  the  target  is 
deterministic . 

Since  the  conditional  probabilities  density  functions 
are  assumed  Gaussian,  we  need  only  to  find  its  mean  and 
variance.   Before  proceeding,  let's  first  define  some 
statistics  for  y  and  x  . 
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Figure    6.2 
Histogram  of    (v-e    )    for   the   image    in   figure    (5.8) 

ir 
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(a)   Means 

To  make  the  formulas  more  general,  we  consider  here 
that  the  random  fields  x(m,n)  and  T(m,n)  are  not  zero-mean. 
Since  the  fields  are  homogeneous,  the  mean  is  a  constant 
and  the  same  for  all  pixels.   All  the  previous  equations  of 
the  image  filters  remain  valid,  provided  we  change  x  to 
(x  -  x)  and  y  to  (y  -  x)  ,  where  x  is  the  mean  value  of  the 
field  x (m,n) . 

We  will  use  the  following  means: 

E(x)   =   x 
E(T)   =   T 


E(v)   =   0 


E(y|H  )  =  yQ  =  x 


E(y|H,)   =   y,   =   ax  +  T 

E(x  )   =   E(x)   -   x 

(b)   Variances 

The  variances  are  given  by 

V(x)   =   c2 

V(xp)   =   ap2 


(6.5) 
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V(T)   =   aT2 


V(v)   =  r 


V(y|H  )   =   a   2   =   a2  +  r 


V(y|H,)   =   a   2   =   a2a2  +  a  2  +  r 
J.       y ,  T 


(6.6) 


(c)   Covariances 

We  have  the  following  covariances 


Cov(v,x)       =      Cov(v,x    )       =      0 


Cov(v,yi)       =      r    ,  i   =   0,1 


Cov(x    ,x)       =      b  (6.7) 


Cov(x    ,y    )       =      Cov(x    ,  x)       =      b 


Cov(x  ,y,)   =   aCov(x  ,x)   =  ab 
(d)   Coefficients  of  Correlation 

The  coefficients  of  correlation  are 


Cov(x  ,x) 

p    -= r 


p  0    0  0    0 


Cov(xp-yo)        b 

po  =  "TV =  7^—  (6"8 

Vy0       p  y0 
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Cov(x  ,  y,  , 

p  J  1  b 

p    =  e =   a  

1       o   a  a    a 

P  Y±  P  Y1 


Now,  returning  to  equation  (6.4)  and  using  the  Gaussian 
pdf ' s ,  we  have 


[y-E(y|x  H  )]2    [y-E(y|x  H  )] 

L(y,x  )   =  ^--E-2 *-£-± (6.9) 

V[y|x  H  ]         V[y|x  H  ] 


where  we  have  dropped  some  constants,  since  they  are  not 
important  in  the  test  against  a  threshold. 
The  regression  of  the  mean  of  y  on  x   is 

V      *    X 

Etylx  H.  ]   =  y.  +  — -   p.(x-x)    i  -   0,1   (6.10) 
-''pi      Ji    a     i   p   p 


Substituting  equations  (6.5)  and  (6.8)  into  (6.10)  gives 


E[y|xpHQ]   =   (l-Y)x  +  yxp 


E[y|x  H. ]   =   a (1-y) x+T+ayx 
J  i    p  1  '         r 


(6.11) 


where  we  have  defined: 


Y   =  -£*  =  —  p  (6.12) 

'      a    2  a      Kp 

P 


Having  the  means  of  the  conditional  pdf ' s ,  we  find 
its  variances : 
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V[y|x  H   ]      =     E{[y   -E(y|x  H    )]     } 
Ljr  '    p  o  Jo      u      p  o 


—      ~      2 

=      E{ [x+v- (l-y)x-yx    ]     } 

IT 


—  —  2 

=      E{ [ (x-x)-y(x   -x)+v]     } 


giving 


i  *  2  2      2 

V[y   x   H]      =      a      +   y    a        +r-    2yCov(x    ,x) 
2  '    p   o  p  p 


Using   equations    (6.7)    and    (6.12)  we   have 


2 
Cov(x    ,x)       =      yo 

P  P 


V[y|x   H    ]      =      a2    -   y<j    2   +   r  (6.13 

J  '    p  o  p 


V[y|x  H    ]      =      E{ [y,-E(y|x  H    ) ]z} 


—  —         ~       2 
=      E{ [ax+T+v-a (1-y) x-T-ayx    ]     } 


—  —  —  2 

=      E{  [a  (x-x)  +  (T-T)  -ay  (x   -x)+v]     } 

P 


and 


2    2  2    2       2  2 

V[y|x  H    ]      =      a   o      -ay    a        +   a        +r  (6.14) 
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Substituting  equations  (6.11),  (6.13)  and  (6.14)  into 
(6.9)  gives 


[y-(l-y)x-yx  ]     [y-a (l-y)x-T-ayx  ] 

L(y,x  )   =      2   2   2^         2,    2      2      I7~      2L 

a  -y  a   +r      a  (a  -y  a   ) +aT  +r 


Let's  define: 


M   =   (l-y)x 


2     2 

N  =   a  -yo 
P 


(6.15) 


/\  ^  /N  /■} 

(y-yx  -M)     (y-ayx  -T-aM) 

L(y,x  )   =   rrr^ ~ 2 -  (6.16! 

J      p         N+r  2^T      ,    2  ■ 

r  a  N  +  r  +  aT 


2       2      ~     2               —     2 
(a  N+r+am  ) (y-yx  -M)  -(N+r) (y-ayx  -T-aM) 
T        P p 

(N+r) (a2N+r+a  2) 


The  denominator  is  a  positive  constant,  because 


2     22       2,,    2.^n 
N  =   a   -  y  a    =  a  d~P   )  >  0 

hr  P 


Dropping  the  denominator  and  simplifying  the  numerator 
of  (6.16),  the  likelihood  ratio  becomes 


L(y,x  )   =   [ (y-x)-y (x  -x) ]   -  6 [ (y-ax-T) -ay (x  -x) ] 

hr  P  hr 

(6.17) 
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where 


N+r (1-Pp2)a2  +  r 

6   "    2TT  7   2    "   "l" 2,  2^  T   2        (6-18) 

a  N+r+aT       a  (1-p   )a  +r+aT 


Given  the  prediction  x  and  the  measurement  y,  the 

decision  rule  consists  in  computing  the  likelihood  ratio 

/\ 
L(x  ,y)  and  compares  it  against  a  threhold  ru  if  greater, 

we  choose  hypothesis  H, ,  if  not  we  choose  hypothesis  H  : 


r 

/\ 

>    n 

. . .  Hypothesis  H, 

L(x  ,y)  < 

<  n 

. . .  Hypothesis  H 

In  figure  6.3  a  block  diagram  of  the  estimator-detector 
filter  is  shown.   It  is  assumed  that  the  background  pre- 
dominates in  the  picture  and  the  objective  is  the  detection 
of  a  smaller  texture  that  we  call  target.   The  background 
texture  is  modeled  as  in  chapter  II.   To  accomplish  such 
modeling,  however,  a  picture  without  target  is  needed.   In 
an  actual  application  it  is  realistic  to  assume  that 
we  have  a  typical  realization  of  this  process  (background) 
before  the  target  enters  in  the  scene.   For 
example,     assume  that  we  are  dealing  with  a  surveillance 
problem,  where  some  area  has  been  scanned  many  times  looking 
for  a  very  low  contrast  target.   Therefore  a  recursive  or 
hybrid  filter  can  be  designed  to  predict  and  estimate  the 
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background.  About  the  target  we  need  only  to  have  a  good 
guess  about  the  mean  and  variance  of  its  gray  level.  In 
cases  of  very  low  contrast  the  mean  may  be  assumed  equal 
to  that  of  the  background.  The  variance,  in  many  cases, 
may  be  assumed  much  smaller  than  that  of  the  background. 
Also,  in  many  cases,  the  observation  noise  is  much  smaller 
than  the  background. 

The  estimator-detector  filter  of  figure  6.3  scans  the 
image  row  by  row  from  top  to  bottom.   Assume  that  the 
previous  pixels  were  correctly  decided  as  belonging  to 
the  background.   The  decision  box  receives  the  measurement 
y  and  the  prediction  x   and  computes  the  likelihood  ratio 
L(y,x  )  from  equation  (6.17)  .  If  this  ratio  is  less  than  a 
chosen  threshold  r\ ,  the  pixel  (m,n)  is  considered  background 
and  the  observation  y  is  used  to  find  the  background  estimate 
x(m,n).   If  the  ratio  is  greater  than  r\ ,    the  pixel  is  con- 
sidered as  belonging  to  the  target  region.   While  in  the 
target  region  the  background  filter  cannot  use  the  observa- 
tion y,  unless  we  have  "a  priori"  knowledge  about  the  target 
gray  level.   Since  we  are  considering  that  the  target  region 
is  small,  the  background  filter  may  run  in  its  prediction 
mode  without  much  degradation,  provided  the  dynamic  model 
is  accurate  enough.   Observe  that  both  recursive  and  hybrid 
predictors  can  be  used,  the  only  difference  being  the  box 
named  "improved  prediction"  that  applies  only  to  the  hybrid 
predictors.   Since  the  three-dimensional  predictors 
presented  in  chapter  V,  makes  use  of  the  measurements  in 
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the  previous  frame,  we  can  expect  a  good  performance  of 
those  filters  in  the  target  region. 

The  gray  level  T(m,n)  of  the  target  texture  can  be 
estimated  using  the  background  prediction  x   and  the 
observation  y|H,  . 


T(m,n)   =  k-1y1   +   k2x 


where 


y,   =  y | H,   =   ax  +  T  +  v 


The  estimation  error  is 


e   =   T-T   =   T-  k, (ax+T+v)  -  k2xp         (6.19! 


In  order  to  choose  an  unbiased  estimator: 


E(e)   =   T  -  k,(ax+T)  -  k_x  =  0 


k2   =  i[T-k1(ax+T) ]  (6.20) 

x 


Using  equations  (6.19)  and  (6.20)  the  weights  k,  and 
k„  can  be  found  using  the  minimum-mean- squared  error  criteria; 

—    2       —2   2—2   2 
axT(a   -b)  +  x  a_  +T*a   ) 

V   =  ]2 T E i  a     o-i) 

1     2—2   2    2        —    2     —2   2—2   2—2      lD"  i; 
a  x   a  +o      -2b)+2axT(a   -b)+x  am  +Tza  ^+x  r 
p  p         T      p 
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Observe  that  if  we  neglect  the  observation  noise 
(r  =  0)  and  the  prediction  error  (x  =x,b=o    =  a  )  , 
equations  (6.21)  and  (6.20)  become 


k.   =   1  ,     k_   =   -a 


thus; 


T   =   y,  -   ax  (6.22 

*  1       p 


Equation  (6.22)  is  intuitively  appealing,  since  it  is 
exactly  what  we  expect  in  the  absence  of  observation  noise 
and  with  an  ideal  predictor.   The  block  diagram  of  figure 
(6.3)  considers  this  simplified  case. 

D.   SPECIAL  CASES 

In  this  section  we  will  study  some  particular  cases  of 
detection.  The  detector  developed  in  the  previous  section 
will  be  applied  to  simple  cases  of  practical  interest  and 

its  performance  (R.O.C.)  will  be  derived. 

2 
1.   Case  I:   a  =  1,  a    =0 


In  this  case  the  target  gray  level  is  deterministic 
and  immersed  in  the  background.   The  target  gray  level 
T  is  an  unknown  constant  and  we  also  don't  know  the  target 
shape  or  location  in  the  picture.   The  objective  is  to 
suppress  the  background  in  order  to  enhance  the  target. 
As  before,  the  recursive  or  hubrid  filters  are  used  to 
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predict  the  background  x  and  the  detector,  using  this 
prediction  and  the  observation  y,  decide  whether  the  pixel 
(m,n)  is  target  or  background.   The  output  is  a  binary 
image  where  1  is  target  and  0_  is  background. 

The  likelihood  ratio,  equation  (6.17),  becomes 


L(y,x  )   =   [(y-x)-y(x  -x)]2  -  6[(y-x-T)-y(x  -x)]2 

P  P  P 


where 


6   =   1 


Thus 


L(y,x  )   -   2T[  (y-x)-y(x  -x)  ]  -T 
P  P 


Dropping  the  constants ,  this  ratio  becomes 


L(y,x  )   =   (y-x)  -  y(x  -x)  (6.23! 

P  P 


where 


a 

Y   =  —  pp 
P 


Equation  (6.23)  is  intuitively  appealing,  because  it  is 
exactly  the  residue  of  the  recursive  filters  when  y  =  1  • 
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This  would  happen  if  the  filters  were  optimum.   Such  a 
decision  rule  with  y  »   1  was  used  in  [5]  as  a  generalization 
of  the  one-dimensional  Kalman  filtering  technique.   However, 
we  have  shown  here  that  such  an  extension  is  not  correct, 
because  the  filters  are  not  optimum,  and,  consequently, 
the  residue  (or  innovation  process  [16])  does  not  correspond 
to  the  likelihood  ratio.   However,  it  is  a  good  approxima- 
tion to  make  y   =  1  for  the  recursive  filters,  in  cases  of 
low  observation  noise  and  highly  correlated  background.   It 
can  be  verified  that  y   =   1  for  the  hybrid  filters. 

Now,  we  will  make  the  assumption  that  y  and  x  have 
a  joint  Gaussian  distribution  in  order  to  find  the  performance 
of  the  detector.   Previously  we  had  assumed  that  the  con- 
ditional pdf ' s  were  normally  distributed  and  we  validated 
such  assumption  with  experimental  results,  but  this  does 
not  mean  that  the  joint  distribution  is  also  Gaussian. 
However,  it  is  analytically  convenient  here,  in  order  to 
have  simple  results. 

If  y  and  x   are  jointly  Gaussian,  the  likelihood 
ratio  L  is  also  Gaussian  and  its  pdf  is 

PT  U)   =  - exp[  (      ?   ]  (6.24) 

u  /2?  aT      2  aT 

ij        Li 

where : 


I     =  y.  -  x      i  =  0,1 
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°L    -   Oy,   +  Y  ^p2  "  2yb  (6.25) 


2     2   2 

a  -  y  a 

YL  P 


2  ,       2   2 
a   +  r  -  v  a 
P 


2   2 
=   (1-p  ^)az  +  r 
P 


The  variable  L  has  the  same  variance  for  both  hypothesis 
H  and  H, ,  only  the  means  are  different  as  given  by 


E(L|H  )   -   £Q   -   (x+v)  -  x   =   0 


E(L|H1)   =   £x(x+T+v) 


-  x   =   T 


In  figure  6.4  the  conditional  pdf ' s  of  the  likeli- 
hood ratio  L(y,x  )  is  shown. 

The  probability  of  false  alarm  is 


Pp   =   prob(L|HQ  >  n) 


i /      exp(-£2/2aT2)d£ 


/2tF  a        n 


—  /    exp(-u2/2)du 

/2? 

n/aL 
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PF   =   erfc(n/aL)  (6.26) 


The  probability  of  detection  is 


Pp  m     prob(L|H1  >  n) 


— /  exp[-(£-T)2/2aT2]d2- 

l  n 


PD   =   erfc(^)  (6.27) 

Jj 


where 


1  9 

erfc(a)   =  /  exp(-u  /2)du 

/2tT 
a 


The  choice  of  the  threshold  can  be  done,  for 
example,  by  specifying  the  maximum  admissible  probability 
of  false  alarm;  in  this  case  we  have  aNeyman-Pearson  detector 
[15] .   Another  criteria  is  to  minimize  the  overall  probability 
of  error: 


p£  =  p(ho)p(e|hq)  +  p(h1)p(e|h1; 


-   P(HQ)PF  +  P(HX)  d-PD) 
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If  we  know  P(H  )  and  P(H,),  an  optimum  threshold 
can  be  found  that  minimizes  P-.   If  this  is  not  the  case, 
the  best  we  can  do  is  to  assume  that  the  hypotheses  are 
equally  likely,  P(H  )  =  P (H.)  =  1/2.   With  this  choice, 
it  can  be  shown  that  the  optimum  threshold  is  n  =  T/2  and 

PE  =  PF  "  1    ~    V 

The  implementation  of  the  estimator-detector  for 

this  case  is  similar  to  the  one  shown  in  figure  6.3.   In 

this  case,  since  we  assume  a  target  with  constant  gray 

level,  and,  also  assuming  neglectable  observation  noise, 

we  might  use  the  measurement  in  the  recursive  estimation, 

while  the  filter  is  in  the  target  region,  provided  that  T 

is  subtracted  from  y | H, .   The  best  estimate  of  T  must  occur 

at  the  edges  of  the  target  region,  because  there,  the 

background  prediction  x   is  the  best. 

2.   Case  II:  a   =   1,  a        f   0 

In  this  case  the  target  gray  level  is  a  random 

_  2 

variable  with  known  mean  T  and  variance  a    and,  as  before, 

we  don't  know  its  shape  or  location  in  the  picture. 

The  likelihood  ratio,  equation  (6.17),  becomes 


L(y,x  )   -   [(y-x)-y(x  -x)]2  -  S[(y-x)-Y(x  -x)-T]2 

¥  hr  P 


=   (1-6)  [  (y-xhy(x  -x)  ]2  +  26T[(y-x)-y(x  -x)  ]-6T2 

P  ir 

(6.28) 
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where 


(1-p  2)a2  +  r 
6   =  E 


2   2      2 

(l-pp^)a/+r+oTz 


and 


X"6   = 2,  2,  I   2   >   ° 

(1-Pp  )cr  +r+aT 


Rearranging  equation  (6.28)  in  order  to  have  a 
quadratic  form,  and  observing  that  (1-6)  is  positive: 


L(y,x  )  _     -   _   o 

(1_6[    -   [(y-x)-y(x  -x) ]   +  26t(y-x)-y(x  -x) 

+  B2  -  3(3  +  T) 


=   [  (y-x)-Y(x  -x)+6]2  -  3(3  +  T) 


Dropping  the  constants,  this  ratio  becomes 


L(y,x  )   =   [(y-x)-Y(x  -x)+S]2  (6.29) 


where 


3    «    T[(l-Pp2)   ^  ^Ey] 


T      T 


o 

^   ""   a   pp 
P  v 
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To  calculate  the  performance  of  the  detector,  we 
make  again  the  assumption  that  y  and  x  have  a  joint 
Gaussian  distribution. 

Define  the  random  variable  Z: 


Z   =   (y-x)  -  y (x  -x)  +  3 

ir 


Since  y  and  x   have  a  bivariate  normal  distribution, 

p 

Z  is  also  normally  distributed  and  its  pdf  is 


_  2 

Pz(z)   =   i exp[~(z"^  ]  (6.30! 

/2?  oz       2az 


z   =   y  -  x  +  6 


2        2     2   2 

a „    =   a    +   y   a     "   -    2p  ■    y    a   a        1=0,1 
Z       y       P      i    y  P 


2     2   2        2      2  2 

=  a   -  y  a    =  a   -pa 
y      p      y    p 


2 

Since  L  =  Z  ,  its  pdf  is 


PT  (2-)   =   —  [P7(/I)  +  P„(-/I)] 
L         2/1    Z         Z 


Thus 


PLU)   =  1 {exp[-(/^-22)2]  +  exp[=^-^-]} 

2az/2FI       20Z  2aZ  1; 
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The  mean  and  variance  of  L(y,x  )  can  be  computed 

P 

using  the  Moment  Generating  Function  of  Z. 


Mz(t)   =   E(ezt)   =   exp[tz  +  |<?z2t2] 


E(L)   =   E(Z2)   -   M,I3tO)   -   z2  +  c  2 


V(L)   =   E(L2)  -  E2(L)   =   E(Z4)  -  E2(Z2) 


=   MZIV(0)  -  [MZII(0)]2 


Therefore 


E(L)   =   z2  +  az2 


V(L)   -   2a„2(a  2  +  2z2) 

u  L 


(6.32) 


The  distance  between  the  means  of  L(y,x  ),  under 

P 

the  hypothesis  H   and  H,  ,  is  calculated  using  (6.30)  and 
(6.32)  : 


d  =   E(L|H1)  -  E(L|H  ) 


E(L|H,)   =   (T+6)2+a2+a  2+r+y2a  2-2p  a   a 
x  i        p     x  y  -|  p 


—    2   2    2     2   2 

=   (T+8)  +o   +am^+r-y  a 
j-       P 


=   (T+B)2+a  2+(l-p  2)a2+r 
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E  L  H  )   =   g   +  a      +  r  +  y  0    -  2p  yo   a 
1  o  '   p       o'  y   D 


,2  ,   2         2   2 

!   +  a   +  r  -  y  a 

P 


2         2   2 

=   6   +  (1-P D  )az  +  r 


thus : 


d   -   T2  +  26T  +  aT2  (6.33) 


From  equation  (6.33)  we  can  see  that  the  distance 
between  the  means  is  non-zero  even  for  a  target  with  zero 
mean. 

The  probability  of  detection  is 

CO 

PD  =  prob(i|H1  >  n)   =   /  p^JH^da 

n 


Using  equation  (6.31): 


CO  , —  2 

Pn     -     i     / exp  [ x~i ]d£ 

u  z  a        /2ttT  2a 

n  zx  zx 


co  . —  —        0 

,  !  "(/I    +    Z ,) 

+  7      /      i exP  [ 5 i Jd£  (6.34) 

a      /TnT  2a 

n  zx  z2 
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Using  the  complementary  error  function  (erfc) , 
equation  (6.34)  becomes 


PD  =  erfc[^-(n1/2-  z1)]+  erfc[^-(n1/2  +  z,)  ] 
z  ~  z 

(6.35: 

Similarly,  the  probability  of  false  alarm  is  given 
by 


P_   =   Prob(5-iH   >  n)   ■    I    PT  (l\E    )di 
c  o  Jj     o 

n 

(6.36) 

P-.  =  erfc[-^-(n1/2  -I  )]  +  erfc[-i-(n1/2 +  1  )] 
r  o  o  o  o 

z  z 

o  o 


—       2 
The  values  of  z  and  a        are  given  by 


2 
zx   =   T+3   -   T[l  +  /j  +  d-Pp2)  /j]     (6.37) 

T  T 

2  2,  2      2 

az     =   (1_pp  )a     aT 

z .  =   6   =  -£*■  +  d-P„2) 


O  a  2        p    „  2 

T 


2  2   2 

a  '  =   (1-p  z)a   +  r 
z  P 

o  ^ 


To  analyze  the  detector  in  terms  of  some  meaningful 
parameters,  let's  define  the  ratios: 
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2 

BNR  =  — (background-to-noise  ratio) 

—2    2 

TBR  =  = — (target-to-background  ratio) 


T2 

RTT   =   — 


2 


Normalizing  the  equations  with  respect  to  the 

2 

background (a   =  1),  equations  (6.35)  and  (6.36)  become 


PD  =   erf c [ 


TBR 
a  1+RTT 


/b+RTT ] 


(6.39) 


+  erfctW -fag-  +    /b+RTT] 

l(a+l+RTT 


PF  -   erf  [*\g  -  /b]  +  erfctlg  +  /b]         (6.40) 


where 


a   "   BNR+  (1  -  Pp2; 


b   =   a  ||£(1  +  RTT) 


(6.41) 


Using  (6.39)  and  (6.40)  we  have  calculated  the 
R.O.C.  (Receiver  Operating  Characteristic)  of  the  detector. 
In  figure  6 . 5  the  effect  of  the  target-to-background  ratio 
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Figure  6.5 

Curve  of  Performance  changing  the 
target-to-background  ratio  (TBR) 
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(TBR)  is  shown.   As  expected,  the  detector  performance  is 
highly  dependent  of  this  ratio.   For  a  probability  of 
false  alarm  of  10%  the  probability  of  detection  varies 
from  67%  to  97%  when  TBR  varies  from  -3  to  3  dB.   In 
figure  (6.6)  the  effect  of  the  background-to-noise  ratio 
is  shown.   Since  this  ratio  is  responsible  for  the  accuracy 
of  the  background  prediction  (x  ) ,  it  is  also  an  important 
factor  in  the  detection.   From  this  figure  we  can  see  that, 
for  a  probability  of  false  alarm  of  10%,  the  probability  of 
detection  increased  from  6  3%  to  9  3%  when  the  BNR  varied 
from  10  to  30  dB. 

In  figure  (6.7)  the  effect  of  the  randomness  of  the 
target  gray  level,  represented  by  the  ratio  RTT  is  shown. 
We  can  also  observe  a  considerable  improvement  in  the 

detection  when  this  ratio  is  increased. 

2 
3.   Case  III:  a   =  0,  a        =0 

In  this  case  the  target  gray  level  is  deterministic 
with  known  value,  though  its  shape  and  location  is  unknown. 
The  target  and  background  regions  are  completely  uncorrelated 
(a  =  0),  thus  it  is  the  case,  for  example,  of  images  in 
the  visible  spectrum  where  there  is  target  or  background, 
but  not  the  addition  of  these  textures . 

The  likelihood  ratio,  equation  (6.17),  for  this 
case,  becomes 


L(y,x  )   =   [(y-x)-y(x  -x)]2  -  6[y-T]2   (6.42) 

hr  ir 
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Figure    6 . 6 

Curves  of  performance  changing  the 
background-to-noise  ratio  (BNR) 


176 


i 
PD 

i 

BNR 

=   10  dB 

1  0\(h  •  ■ 

TBR   =   0   dB 

■«■ 

RTT 

=  100 

■«" 

Oi     "7C 

RTT  =  10 

RTT  =  1 

Ql     ^ 

ft  C*0l  — * 

1 

> 

1 

i 

1 

1 

i 

' 

e  <d 


e  2 


0  4 


0  6 


e  8 


l  a  p. 


Figure  6 . 7 
Curves  of  performance  changing  the  ratio  RTT 
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*  1/  2     2.  , 

6  =  -(a  -yap  +r) 


ln    2,  2., 
?(1-Pp  )a  +1 


Let's  find  the  distance  between  the  means,  under 
the  two  hypotheses  H   and  H,: 


/S  /")  r\  /\  >-)  /\  f\ 

L(y,x    )       =       (y-x)    +Y     (x_-x)    -2y (x   -x) (y-x)-6 (y-T) 

hr  hr  hr 


y   H        =      x   +   v 

J  '    o 


E(L|H  )   =   a2+r+Y2a  2-2yb-6E [ (y-x) -  (T-x) ] 2 


=   a  +r-y  a   -5 (a  +r)-6(T-x) 

ir 


E(L|H  )   =   (1-6) (a2+r)  -  y2J  2    -  6 (T-x) 2     (6.43) 


y  j  H,   =  T  +  v 


E(L|H,)   =   (T-x)2  +  (l-6)r  +  Y2a  2 


(6.44) 


Subtracting  (6.43)  from  (6.44)  we  find  the  distance 


d   =   E(L|H1)  -  E(L|H  ) 


d   =   (1+6) (T-x)2  +  (2p  2+6-l)a2 

hr 


(6.45) 


Observe  that  the  distance  is  non-zero,  even  for  tar- 
gets with  the  same  gray  level  as  the  mean  of  the  background 

(T»X). 
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The  performance  of  the  detector  can  be  found, 
provided  we  assume,  as  before,  a  joint  Normal  distribution 
for  y  and  x  ,  however  it  is  not  possible  to  have  an  elegant 
analytical  solution  as  in  the  other  cases. 
4.   Case  IV:   a  =  0,  aT  fi   0 

This  case  might  cover  the  model  of  some  images  with 
two  quite  distinct  textures.   A  difficult  problem,  for 
example,  is  the  restoration  of  pictures  with  two  different 
regions,  one  is  the  sky  and  the  other  the  sea.   This  is  a 
typical  case  where  we  cannot  model  the  image  as  a  homogeneous 
random  field.   One  approach  to  this  problem  is  to  model  each 
texture,  separately,  as  two  homogeneous  random  fields,  by 
the  method  of  Chapter  II,  and  design  one  image  filter  for 
each  texture.   To  apply  these  filters,  first  we  have  to 
decide  to  which  texture  the  pixel,  to  be  processed,  belongs. 
Such  a  decision  can  be  made  by  the  detector  of  the  present 
case.   In  figure  (6.8)  the  block  diagram  for  this  situation 
is  shown.   Following  the  same  notation  as  before,  we  have 
a  filter  for  the  background  x  (one  texture)  and  another 
filter  for  the  target  T  (the  other  texture) .   The  threshold 
decision  selects  the  proper  filter,  while  the  other  filter 
runs  in  its  prediction  mode. 

The  likelihood  ratio  for  this  case,  equation  (6.17), 
is 


L(y,x  )   =   [(y-x)-y(x  -x) ]2  -  6 (y-T) 2   (6.46) 
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6   = 


2   2 
(1-p _  )a  +r 

ir 


Let's  compute  the  distance  between  the  means: 


E(LJH  )   =   (1-6) (a2+r)-y2a  2-6 (T-x) 2 


E(L|H1)   =   (1-6) (aT2+r)  +  y2a  2  +  (T-x)2 


d   =   E(L|H1)  -  E(L|H  ) 


d  =   (1+6)  (T  -  x)2  +  (1-6)  (aT2-a)  +  2pp2a2   (6.47. 


Equation  (6.47)  shows  clearly  the  differences 
between  the  two  textures  (mean  and  variance) . 

As  in  Case  III,  the  performance  of  this  detector 
can  be  found  using  the  Gaussian  hypothesis,  however  it  is 
not  possible  to  have  an  elegant  analytical  solution. 
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Figure  6 . 8 
Estimator-detector  filter  for  two  textures 
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VII.   TARGET  TRACKING 

A.   INTRODUCTION 

In  the  previous  chapters  we  have  developed  statistical 
image  filters  that  exploit  spatial  and/or  temporal  redun- 
dancies existing  in  the  pictures.   One  application  of  those 
filters  is  the  restoration  of  images  contaminated  with 
observation  noise.   In  this  application,  the  objective  is 
to  remove  the  observation  noise,  in  order  to  recover  the 
original  image.   Another  application  is  to  extract  some 
desired  information  contained  in  the  pictures,  which  we  have 
called  target.   In  this  case,  we  want  to  suppress  the  rest 
of  the  information,  which  we  have  called  background,  in 
order  to  enhance  the  target. 

The  target  detection  and  tracking  problem,  by  means 
of  pictures  sequenced  in  time,  may  be  divided  into  three 
phases : 

(a)  Extraction  of  the  targets  from  the  background, 
creating  a  binary  picture  (1-target,  O-background) . 

(b)  Recognition  of  the  target (s)  that  we  are  interested 
in  tracking. 

(c)  Track  the  target (s)  from  frame  to  frame. 

This  research  addressed  the  first  and  third  phases. 
The  recognition  problem,  for  our  present  purpose,  will  be 
considered  as  solved  by  a  human  operator,  or,  in  the  case 
of  an  automatic  system,  by  some  existing  algorithms  118] , 
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[19]  of  pattern  recognition.   The  extraction  problem  was 
covered  by  the  previous  chapters . 

In  this  chapter  we  address  the  tracking  of  targets 
from  frame  to  frame.   The  basic  idea  is  to  model  the  target 
movement  within  the  pictures,  and  to  construct  a  Kalman 
filter  to  track  the  target  centroid,  using  the  image  filters 
as  the  measurement  device.   Of  course,  others  relevant 
points  of  the  target  might  be  tracked,  in  a  similar  way, 
in  order  to  find,  for  example,  the  target  attitude.   The 
image  filters  referred  to  here  are  the  three-dimensional 
recursive  and  hybrid  predictors,  presented  in  Chapters  IV 
and  V,  respectively,  and  the  detector  developed  in  Chapter 
VI. 

B.   TARGET  TRACKING  FILTER 

In  this  section  we  will  construct  a  Kalman' filter  to 
track  the  target  centroid.   Assume  that  we  receive  pictures 
at  some  specified  rate,  say  30  pictures/sec.   Since  the 
pictures  have  only  two  dimensions,  the  target-image  dynamics 
is  related  only  to  the  components  of  the  actual  movement 
that  are  parallel  to  the  picture-plane  (see  figure  7.1) . 
The  effect  of  the  movement  perpendicular  to  the  picture  is 
to  increase  or  decrease  the  size  of  the  target-image.   If 
the  target  has  the  same  aspect  at  all  angles  (for  example : 
a  sphere) ,  then  it  is  possible  to  compute  such  component  , 
using  successive  frames. 
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The  target-image  becomes  a  point-target  at  large  dis- 
tances, but,  in  general,  it  is  a  mass-target  with  some  shape 
which  changes  when  it  maneuvers.   Such  change  can  have  very 
interesting  applications  in  the  target  tracking  problem, 
since  it  gives  useful  information  of  target  maneuvering. 

In  what  follows,  we  model  the  movement  of  the  target 
centroid  in  the  plane  of  the  picture  and  construct  a  Kalman 
filter  to  estimate  its  x  and  y  coordinates,  and  also  to 
predict  its  position  in  the  next  frame. 

X 


TARGET- IMAGE 


TARGET 


PICTURE- PLANE 


Figure  7.1 
Target  dynamics 
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1.   Dynamic  Model 

Since  we  don't  know  the  intention  of  the  target 
driver,  we  will  consider  two  random  accelerations  in  the  x 
and  y  directions,  therefore  the  position  of  the  target- 
image  is  given  by 


x(k+l)   =   x(k)  +  x(k)T  +  ■iw1(k)T2 


y(k+l)   ■   y(k)  +  y(k)T  +  -|w2(k)T2 


(7.1) 


Equation  (7.1)  assumes  that  the  time  separation  T 
between  frames  is  small  enough  such  that  we  can  consider 
that  the  acceleration  and  velocities  are  constants  between 
frames . 

The  velocities  are  given  by 


x(k+l)   =   x(k)  +  wx(k)T 


yC.k+1)   -  y(k)  +  w2(k)T 


(7.2) 


where  we  have  used  in  this  derivation  the  simplified 
notation  x(k)  for  x(kT). 
To  use  state  variable  notation,  define  the  state 
vector  x  and  the  random  forcing  input  w: 
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X   = 


X 

X 

y 
y 


w   = 


w. 


w. 


2 


In  state  space  notation,  equations  (7.1)  and  (7.2', 


become 


x(k+i; 


1 
0 
0 

0 


T 
1 

0 
0 


0 
0 

1 

0 


0 
0 


x(k) 


T2/2 


T 
0 


T2/2 


w(k) 


(7.3) 


The  random  forcing  input  w  is  considered  zero-mean 
white  noise  uncorrelated  with  x,  and  has  the  covariance 
matrix  below 


E[w(k)wx(k)J   =   Q   = 


(7.4) 
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2 .   The  Observation  Vector 

The  centroid  of  the  target-image  is  measured  by 
the  image  filters  previously  developed.   The  pixels  are 
examined  one  by  one  and  classified  as  target  or  background. 
After  this  classification  the  target  centroid  is  computed 
by 


x 

m 


N  N 

=  k      I    nkT(mk'nk}  '     S   =    I    T(lW 
k=l  k=l 


N 

^m  =   I  I   mkT(mk'nk)  (7'5) 

k=l 


where  (m,  ,n,  )  are  the  coordinates  of  the  pixels  classified 
as  target.   If  we  create  a  binary  picture  with  T (m,  ,n,  )  =  1, 
equations  (7,5)  become 


N 

1   V 
m     N  L      k 

k=l 

(7.6! 
N 


1   r 

^m        NT    L 

k=l 


m     N  L      k 


Since  the  image  filters  are  not  perfect,  some 
target-pixels  are  missed  and  some  background-pixels  are 
taken  as  targets.   Therefore  the  measurement  of  the  centroid 
has  an  error : 
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xm   =   X  +  V, 

m  1 


y    =   y  +  v„ 


(7.7) 


where  v,  and  v~  are  the  errors  in  the  x  and  y  coordinates, 
respectively. 

Let's  analyze  the  errors  v,  and  v_ .   First  deter-- 
mine  if  they  are  uncorrelated.   In  figure  7.2  assume  that 
the  image  filter  missed  the  target-pixels  in  region  A. 
The  result  is  an  error  in  the  observation  of  the  centroid. 
Both  x  and  y  coordinates  will  be  in  error.   In  other  situa- 
tions only  one  coordinate  is  wrong.   In  general,  we  can  not 
draw  any  inference  about  one  error  by  the  knowledge  of  the 
other,  therefore  we  will  assume  that  v,  and  v2  are  uncorrelated 


m 


m 

A 
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Vc 

c 

TARGET 

r 

Figure  7.2 
Measurement  error 
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We  must  determine  if  the  errors  are  correlated  with 
the  coordinates  (x,y) .   Since  the  background  and  the  target 
gray  levels  are  considered  homogeneous  random  fields, 
there  is  no  reason  to  think  that  the  image  filter  will  be 
more  likely  to  make  errors  for  some  positions  of  the 
target  within  the  frame,  or  at  some  special  frames.   On 
the  other  hand,  the  nature  of  the  filter  itself  could  be 
a  source  of  correlation.   Both  image  filters  (recursive 
and  hybrid)  use  recursion  in  space,  therefore  the  background 
prediction  is  poor  near  the  top  left  corner  and  also  near 
the  first  row  and  column  of  all  frames,  because  of  the 
boundary  transients  of  the  filters.   In  the  first  frame  the 
hybrid  predictor  does  not  make  use  of  the  time  correlation 
in  this  frame.   In  the  recursive  predictor  the  first  few 
frames  have  transients  in  recursion  in  time.   The  conclusion 
is  that  there  are  some  positions  (x,y)  in  the  time-frame 
that  have  poor  probability  of  detection.   However,  we  have 
seen  that  the  recursive  filters  converge  very  fast,  there- 
fore, we  will  neglect  such  correlation,  since  it  occurs  only 
in  a  very  small  region. 

One  must  test  to  see  if  the  errors  are  zero-mean. 
The  detection  error  could  happen  at  any  pixel,  but  the  bias 
is  most  likely  in  the  direction  that  the  picture  is  scanned 
(top  left  corner  to  right  bottom  corner) ,  because  when  the 
filter  is  in  the  target  region  the  background  predictors, 
in  general,  can  not  use  the  observation  y(m,n),  which 
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results  in  a  degradation  in  the  detection.   However,  we 
expect  that  such  degradation  can  be  neglected  for  practical 
applications  where  the  target  has  small  size  and  the  back- 
ground correlation  is  high.  There  are  other  schemes  for  pro- 
cessing that  can  be  done  to  overcome  this  problem.   One  is 
for  example,  to  process  the  pictures  top-to-bottom  and 
bottom-to-top,  alternatively,  from  frame  to  frame. 

After  these  considerations  we  conclude  that  it  is 
reasonable  to  consider  the  errors  v..  and  v~  zero-mean  white 
noise  and  uncorrelated  with  x  and  y.   Such  conclusions 
simplify  considerably  the  following  derivation  and  the 
result  will  be  a  very  simple  Kalman  filter. 

Define  the  measurement  and  error  vectors  as  below 


Y   = 


x 


m 


m 


v   = 


v. 


V, 


(7.8; 


In  state  space  notation,  equation  (7.7)  becomes 


y(k) 


10    0    0 


0    0    10 


x(k)  +  v(k) 


(7.9) 


where  v(k)  is  a  zero-mean  stationary  random  process,  uncor- 
related with  x(k)  and  with  the  covariance  matrix  below 
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E[v(k)vx(k)]   =   R  - 


R,     0 


(7.10) 


3.   Kalman  Filter 


The  dynamic  model  is 


where 


x(k+l)   =   $  x(k)  +  r  w(k) 


(7.11) 


1 

T 

0 

0 

0 

1 

0 

0 

0 

0 

1 

T 

0 

0 

0 

1 

(7.12) 


r  - 


V    o 


T_ 

2 


(7.13: 


X   = 


y 
y 


L 


W   = 


W, 


LW2 


(7.14; 
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The  measurement  model  is  given  by 


y(k)   =  H  x(k)  +  v(k) 


(7.15) 


where 


H  = 


0     0 


:7.i6) 


Y   = 


m 


■m 


v   = 


Lv2 


.7.17) 


The  random  forcing  input  w(k)  and  the  measurement 
noise  v(k)  are  considered  white  noise  uncorrelated  between 
themselves  and  also  with  the  states.   Its  covariance  matrices 
are  given  by 


Q  = 


Qx    0 


R  = 


R1     0 


R, 


(7.i8: 


The  Kalman  filter  equations  are  [17] : 


x(k)   =  x' (k)  +G(k)[y(k)  -Hx'(k)]     (7.19) 
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where  x(k)  is  the  estimated  state  vector  and  x' (k)  is  the 
predicted  state  vector  given  by 

x' (k)   -   $  x(k-l)  (7.20) 

G(k)  is  the  filter  gain,  computed  as  below  [17] : 


G(k)   =   P'(k)  HT[HP' (k)HT+R]_1  (7. 2i: 


P(k)   =   [I-G(k)H]P'  (k)  (7.22) 


P'(k+1)   =   *P(k)$T  +  rQTT  (7.23) 


where  P (k)  is  the  covariance  matrix  of  the  state  error 
and  P' (k)  is  the  predicted  covariance  matrix  of  the  state 
error. 

The  estimated  state  error  vector  is 

e(k)   -   x(k)  -  x(k)  (7.24) 

The  covariance  matrix  of  the  state  error  is 


P(k)   -   E[e(k)eT(k)]  (7.25'. 


The  predicted  state  error  vector  is 


e' (k)   =   x' (k)  -  x(k)  (7.26) 
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The  predicted  covariance  matrix  of  the  state  error  is 


P" (k)   =  E[e' (k)e,T(k) ] 


(7.27; 


The  gain  G(k)  is  computed  by  initialization  of 
equation  (7.21)  with  P'(l)  to  have  G(l),  then  find  P(l) 
by  equation  (7.22)  and  P'(2)  by  equation  (7.23).   Following 
this  procedure  the  matrices  G(k),  P(k)  and  P'(k)  are 
calculated. 

Since  the  x  and  y  coordinates  of  the  target  centroid, 
as  well  as  its  measurement  errors  v,  and  v~ ,  are  uncorre- 
cted, we  can  initialize  equation  (7.21)  with  the  predicted 
covariance  of  the  state  error  below 


P'(l)   = 


P^d)    P£2(l)    o 


Pi2(i: 


p22(i; 


p1   (i)        p  * 

33l±j      34^' 


p34(1)     P44(i; 


(7.28) 


With  such  initialization  the  matrices  G(k)  and 
P(k)  have  the  forms: 
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G(k)   = 


gx(k) 


g2(k) 


g3(k) 


940<> 


(7.29) 


P(k)   - 


Pll(k)     P12(k)     ° 


P12(k)     p22(k)     ° 


0  P33(k) 


0  P34(k) 


P34(k) 


p44(k: 


:7.30) 


Working  out  the  equations  (7.21)  through  (7.23. 
the  results  are 


gx(k) 


pii(k) 


Ri  +  pii(k) 


P12(k) 
g2(k)   =   Rx  +  P[1(k: 


g3(k) 


P33(k) 


R2  +  P33(k) 


(7.3i: 


g4Ck) 


p34(k) 


R2  +  P33(k) 
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Pu(k)   =   [l-g1(k)]Pj_1(k) 


P12(k)   =   [l-g1(k)]P;|_2(k) 


P22(k)   =   P^2(k)  -  g2(k)Pj2(k) 


P33(k)   =   [l-g3(k)]P^3(k) 


P34(k)   =   [l-g3(k)]P34(k) 


p44<k>    -    p44(k)-g4(k)p34(k: 


Pj_1(k+1)   =   P11(k)+2TP12(k)+T2P22(k)+jT4Q1 


(7.32) 


p{2(k+i; 


P12(k)4-TP*2(k)4T3Ql 


P22(k+1) 


P22(k)+T  Q 


P'   (k+1)   = 


P33 (k) +2TP34 (k) +T2P44 (k) +iT4Q2 


(7.33: 


P'4(k+1)   =   P34(k)+TP44(k)4T3Q2 


P'4(k+1)   =   P44(k)+T  Q2 
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Initializing  the  filter  equation  with  the  initial 
prediction  x' (1) ,  we  have  the  estimated  state  vector 


x 


(k)   =  x'  (k)  +  g]_(k)  [xm(k)-x' (k)] 


x(k)   =  x'(k)  +  g  (k) [x  (k)-x' (k)] 

J2     m 


y(k)   =y'(k)  +  g3 (k) [ym(k) -y' (k) ] 


y(k)   =  y'(k)  +  g4(k)  [ym(k)-y'  (k)  ] 


The  predicted  state  vector  is 


x' (k+1)   =   x(k)  +  T  x(k) 


x'  (k+1)   =   x(k) 


(7.34) 


(7.35 


y'  (k+1)   =  y(k)  +  T  y  (k) 

y' (k+1)   =  y(k) 

In  figure  7.3  we  show  a  flowchart  for  implementation 
of  the  filter. 
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P'  (1) 

I       m  I       m  _l 

G(k)    =    P    H    [HP    H   +R] 


P(k)     =     [I-GH]    P     (k) 


x(k)    =   x    (k)  +G(k)  [y(k)  -  HX    (k)] 


Figure  7.3 
Kalman  filter  flowchart 
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C.   FILTER  APPLICATION 

The  tracking  filter  developed  in  this  chapter  is  used, 
primarily,  to  provide  the  sensor  servo  with  the  predicted 
centroid  coordinates  for  the  next  frame.   Besides  this 
application,  the  information  of  the  predicted  position  of 
the  target  can  be  used: 

(a)  To  reduce  the  image  processing, 

(b)  To  improve  the  performance  of  the  target  detector. 
The  reduction  in  the  image  processing  can  be  achieved 

by  delimiting  to  a  small  area  to  be  processed,  centered 
at  the  predicted  position  of  the  target  centroid.   Instead 
of  processing,  for  example,  a  100x100  picture  we  might 
work  only  in  a  20x20  region,  depending  on  the  target  size. 
With  such  reduction  we  could  afford  to  make  more  processing 
in  this  smaller  area,  in  order  to  improve  the  performance 
of  the  target  detector. 

Some  examples  of  this  extra  processing  are: 
(a)   Scan  the  picture  in  two  directions,  one  starting 

at  the  top  left  corner  and  the  other  at  the  bottom 
rxght  corner.   The  results  are  two  binary  pictures 
(1-target,  0-background)  with  greater  probability 
of  error  for  the  pixels   located  in  the  transient 
region  of  the  recursion  in  space  (see  figure  7.4). 
One  simple  way  to  combine  the  two  estimates  is  to 
pick  the  half  of  each  picture  opposite  to  the  starting 
point,  because,  in  this  way  we  remove  most  of  the 
unfavorable  region  of  detection. 
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If  more  processing  is  allowable  one  might  scan  the 
picture  four  times  starting  at  each  corner  and  pick 
the  quarter  of  each  estimated  picture  opposite  to  the 
starting  point.   In  this  case  the  detection  is 
completely  free  of  the  transient  region. 
(b)   In  the  detection  of  the  target  edges  the  direction 
of  scanning  the  picture  is  very  important.   Since  a 
good  prediction  of  the  background  is  needed  and,  in 
general,  such  prediction  is  degraded  while  in  the 
target  region,  the  result  is  that  the  detection  of  the 
edge  at  the  left  is  better  than  the  one  at  right 
(see  figure  7.5).   One  way  to  overcome  this  problem 
is  to  use  the  estimator-detector  only  in  favorable 
conditions.   Assume  that  the  prediction  of  the  target 
centroid  is  good.   Process  first  the  left  hand  part 
of  the  picture  until  the  line  AB  (see  figure  7.5) . 
Next,  process  the  right  hand  part,  starting  at  the 
top  right  corner.   In  this  way,  the  filter  will  work 
in  more  favorable  conditions,  since  it  will  face  both 
left  and  right  edges  with  good  background  predictions. 
The  implementation  of  such  a  scheme  may  be  simplified 
if  we  invert  the  columns  of  the  right  hand  part  before 
and  after  the  processing.   In  this  way,  the  filter 
estimates  the  left  hand  part  and  it  is  reinitialized 
at  the  boundary  (line  AB)  to  proceed  to  the  estimation 
of  the  inverted  right  hand  part.   Therefore,  the  only 
additional  process  would  be  the  required  inversion. 
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VIII.   COMPUTER  SIMULATION 

A.   DESCRIPTION  OF  THE  SYSTEM 

A  computer  simulation  of  the  target  tracking  and  detection 
problem  was  implemented.   The  block  diagram  of  the  system  is 
presented  in  figure  8.1.   It  is  composed  of  three  parts: 
time-frame  generation,  image  processing  and  target  tracking. 
The  generator  of  the  time-frame  creates  images  at  some  speci- 
fied rate  and  characteristics.   The  images  contain  the  target 
and  are  contaminated  with  additive  white  Gaussian  noise.   The 
target  has  a  random  movement  from  frame  to  frame.   The  images 
are  processed  in  two  modes:   search  and  window.   The  search 
mode  is  used  to  acquire  the  target.   In  this  mode  the  whole 
picture  is  processed  in  order  to  extract  the  target  from  the 
background.   It  is  assumed  that  a  recognition  phase  exists 
to  select  the  desired  target.   In  this  simulation  we  have 
used  only  one  target  to  avoid  such  a  phase.   After  the 
acquisition  phase,  the  images  are  processed  only  in  a  window 
centered  at  the  predicted  position  of  the  target  centroid. 
The  output  of  the  image  filter  is  the  measurement  of  the 

coordinates  (a  ,6  )  of  the  target  centroid.   The  target 
mm 

tracking  filter  receives  these  measurements  and  updates 
the  estimation  of  the  target  centroid  (a, 3)  and  also 
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computes  the  prediction  for  the  next  frame  (n  <gD )  which 
is  feedback  to  the  image  filter. 

More  details  about  each  block  of  figure  8.1  are  given 
in  the  rest  of  this  section. 

1.   Time-frame  Generation 

The  background  has  the  autocorrelation  function: 


R(i,j,k)   =  a   pvl1!  p^3'  pt'  I    i,j/k  =  0,±1,±2, 


The  dynamic  model  that  generates  a  random  field 
with  such  an  autocorrelation  function  is 


x(m,n,t)   =  A  x  +  w(m,n,t) 


where: 


A   =   [p   P,  P^  ~P  p,  ~P  P,_  -P,  P,.  P  P^Pj.] 
—       v  h  t   v  h   v  t   h  t   v  h  t 


T 
x   =   [x(m-l,n,t)  x(m,n-l,t)  x(m,n,t-l)  x(m-l,n-l,t) 


x(m-l,n,t-l)  x(m,n-l,t-l)  x (m-1 ,n-l, t-1) ] 


E[w2(m,n,t)]   =   Q   =   a2 (1  -  py2)  (1  -  ph2)  (1  -  pt2) 


The  block  "Background  generator"  implements  this 
dynamic  model  by  driving  the  equation  with  zero-mean  white 
Gaussian  noise  and  variance  Q. 
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The  block  "Target  Generator"  creates  a  binary  image 
(T  -  target,  0  -  background) .   The  target  has  an  arbitrary 
shape  and  it  is  centered  at  the  location  (x,y)  given  by  the 
block  "centroid  movement".   Such  movement  is  generated  using 
the  dynamic  model  of  equation  (7.1) . 

The  block  "noise  generator"  creates  an  image  with 
zero-mean  white  Gaussian  noise  at  some  specified  variance. 

Then,  those  three  images  are  added  in  order  to 
simulate  the  situation  addressed  in  case  I  of  chapter  VI. 


y  H   =  x  +  v 

1  '  o 


y  [H,   =   x  +  T  +  v 

where: 

x   -   background  gray  level 
v   -  observation  noise 
T   -   target  gray  level 

2 .   Image  Processing 

The  three-dimensional  recursive  and  hybrid  filters 

were  implemented  in  order  to  predict  the  background  gray 

level  x  .   The  decision  rule  using  x   and  the  observation 
P  y   P 

y  is 


L(y,x  )   =  y  -  y  xp 
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L(y,x  )  i 


>  n 
<  n 


Hypothesis  H, 
Hypothesis  H 


The  block  diagram  of  the  combined  estimator-detector 
filter  is  presented  in  figure  6.3,  particularized  to  case 
I  (a  =  1,  aT2  =  0) . 

The  constant  y  was  defined  (see  equations  6.7 
and  6.12)  by 


cov (x,x 


Y   = 


J 

a   pp 
P 


where : 


ap2      =      E[xp2]       =      E[xp(x+ep)] 


=      cov(x,x   )    +  E[x  e    ] 

P  P  P 


For  the  recursive  filter  the  prediction  error  e 

P 

is  not  orthogonal  to  x  ,  because  the  filter  is  not  optimum. 

However,  the  hybrid  filter  is  such  that  the  prediction  error 

e  is  orthogonal  to  x  ,  therefore: 
P  P 


Y   =  < 


for  the  hybrid  filter 


for  the  recursive  filter 
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The  threshold  n  is  chosen  such  that  the  probability 
of  a  false  alarm  is  below  some  desired  value.   From  equation 
(6.26) 


Pp   =   erfc  (-£-) 
F  GL 


For  n  =  5aT    P_,  =  1.5x10 
L    F 


In  the  simulation  the  threshold  can  be  specified  by 
the  operator  or  automatically  set  at  the  previous  value 


n   -   5aL  =   5 


2   2 
(1-p   )a   +  r    (see  equation  6.25) 
P 


Observe  that  this  criteria  is  completely  independent 
of  "a  priori"  knowledge  of  the  target  gray  level. 
We  have  defined  the  following  ratios: 


TBR.   =   Target-to-background  ratio  at  input 
TBR   =   Target-to-background  ratio  at  output 

BNR   =   Background-to-noise  ratio 

TBR 
P.G.   =    p   =   Processing  gain 

T2 
TBRi   =   -*■= (8.1) 

a  +  r 


[l|hJ 


T2 
TBR    =   -   L  2      "   —2  (8-2) 
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P.G.   =   Hi±r   =    BNR  +  1 (8_3; 

aT^       (1  -  p   )BNR  +  1 
h  p 

2 

BNR    =   — 


From  (8.3)  we  can  see  that  one  upper  bound  for 
P.G.  is 

P.G.   <_  BNR  +  1 

Therefore,  the  observation  noise  has  to  be  well 
below  the  background  in  order  to  have  a  substantial  target 
enhancement.   However,  in  many  applications,  the  observation 
noise  is  really  neglectable  when  compared  with  the  back- 
ground and,  therefore,  substantial  target  enhancement  (or 
background  suppression)  can  be  achieved  with  such  a  method. 

After  the  detection  of  each  target-pixel,  the 
centroid  is  computed  by 


N 


m 


1  v 
i=l 


N 

3    =  h      I   m. 
fit      N  L      1 

i=l 


where  (m.,n.)  are  the  coordinates  of  the  i  target-pixel, 
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3.   Target  Tracking  Filter 

The  target  tracking  filter  is  the  implementation  of 

the  Kalman  filter  constructed  in  Chapter  VII.   It  receives 

the  target  centroid  measurement  (a  ,3  )  from  the  image  filter 
3  m  m  J 

and  computes  the  estimated  position  (a  ,6),  as  well  as  the 
predicted  position  in  the  next  frame  (a  ,3  ).   The  filter 
gains  are  computed  on-line  according  to  the  flowchart  of 
figure  7.3. 

B.   SELECTED  RESULTS 

In  this  section  we  present  some  relevant  results  of 
the  simulation.   The  pictures  have  the  size  34x70,  compati- 
ble with  the  Tektronics  4012  Display.   The  images  are 
pictorially  displayed  with  8  levels  of  quantization  obtained 
by  superimposing  some  characters.   The  target  has  a 
"diamond"  shape,  but  it  can  be  easily  changed  to  other 
shapes  enclosed  by  a  7x7  matrix.   In  the  tracking  mode 
the  image  processing  is  only  applied  to  a  20  x  20  window 
centered  at  the  predicted  centroid  of  the  target. 

1.   Background  Prediction 

Since  the  decision  rule  is  highly  dependent  on  the 
background  prediction,  we  have  measured  the  prediction 
error  for  several  situations  and  compared  with  the  theoreti- 
cal results.   The  results  are  presented  in  tables  8.1 
through  8.3.   Observe  that  the  theoretical  and  experimental 
results  have  the  same  order  of  magnitude.   We  can  also 
observe  that  the  experimental  results  are  always  greater 
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TABLE    8 . 1 


NOISE 
VARIANCE 

3D- RECURSIVE 
PREDICTOR 

3D-HYBRID 

PREDICTOR 

THEORY 

EXPERIMENT 

THEORY 

EXPERIMENT 

1.0 
0.8 

0.6 
0.4 
0.2 

.293         .306 
.267         .280 
.236         .255 
.197         .216 
.144         .166 

.282         .360 
.258         .332 
.229         .308 
.195         .266 
.150         .206 

PICTURE  VARIANCE   =   1.0 
Pv  =  Ph   =   0.7 
pt   =   0.95 

TABLE    8 . 2 


NOISE 
VARIANCE 

3D- RECURSIVE 
PREDICTOR 

3D- HYBRID 

PREDICTOR 

THEORY 

EXPERIMENT 

THEORY 

EXPERIMENT 

1.0 
0.8 
0.6 
0.4 
0.2 

.351         .364 
.324         .343 
.293         .316 
.251         .266 
.191         .207 

.303         .353 
.283         .329 
.260         .295 
.231         .269 
.192         .217 

PICTURE  VARIANCE   =   1.0 
pv  =  ph   =   0.8 
Pt   =   0.8 
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TABLE  8 . 3 


NOISE 
VARIANCE 

3D-RECURSIVE 
PREDICTOR 

3D-HYBRID 

PREDICTOR 

THEORY 

EXPERIMENT 

THEORY 

EXPERIMENT 

1-0 
0.8 
0.6 
0.4 
0.  2 

.376         .388 
.349         .353 
.317         .337 
.274         .294 
.213         .231 

.338         .378 
.317         .352 
.292         .330 
.260         .295 
.216         .244 

PICTURE  VARIANCE   ■   1.0 

p   =  p,   =   0.8 
v     h 

Pt   -   0.7 

than  the  theoretical  ones.   Such  discrepancy  is  due  to  the 
fact  that  the  theoretical  values  are  computed  for  the 
steady-state  condition,  but  the  experimental  values  are 
the  result  of  averaging  over  a  small  ensemble  (20  frames) 
and  small  images  (34x70),  which  means  a  reasonable  influence 
of  the  transient  of  the  image  filters. 

Therefore,  we  consider  that  these  results  are  a 
reasonable  validation  for  the  theoretical  methods  of  filter 
design  and  performance  evaluation  presented  before. 
2 .   Transition  Region 

To  observe  the  transient  of  the  image  filters,  we 
have  applied  the  decision  rule  to  a  time  sequence  of  images 
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(time-frame)  without  target.   The  threshold  was  set  at 
n  =  0.35.   The  characteristics  of  the  images  are: 


2 
a   =1,    pv  =  ph  =  0.93,    p   =  0.95,    BNR  =  30  dB 


The  results  for  the  recursive  filter  are  shown  in  figures 
8.2  through  8.5.   As  mentioned  before,  the  3D-recursive 
filter  gives  degraded  predictions  at  the  first  few  frames 
and  near  the  first  row  and  column  for  all  frames,  due  to 
the  time  and  space  recursion,  respectively.   Observe  that 
in  frame  2  (figure  8.2)  the  false  alarms  occurred  mostly 
near  the  first  column  and  row,  but  they  can  also  be  seen 
in  the  rest  of  the  picture.   In  frame  10  (figure  8.4)  the 
situation  is  better,  because  it  the  steady-state  was  reached 
in  time,  but  the  false  alarms  persist  due  to  the  transient 
in  space.   In  chapter  VII  we  have  suggested  some  methods 
to  reduce  such  undesirable  effects  of  the  transition  region. 
In  this  simulation  we  have  used  a  simple  method  that  is 
to  increase  the  threshold  for  the  pixels  at  the  first  5 
rows  and  columns.   The  results  of  such  a  method  are  shown 
in  figures  8.3  and  8.5.   Observe  the  substantial  improvement 
in  both  frames  2  and  10.   The  false  alarms  of  frame  2 
(figure  8.3)  are  basically  due  to  the  transient  in  time. 
The  percentage  of  false  alarm  after  frame  5 ,  as  in  frame 
10  (figure  8.5),  is  quite  close  to  the  theoretical  value. 
Of  course,  the  probability  of  detection  is  reduced  in  the 
transient  region,  due  to  the  higher  threhold,  however  this 
region  is  very  small  for  the  usual  size  of  pictures  (100x100! 
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Figure    8.2 


False   alarms   of   the   recursive 
filter   at   frame    2,    using 
one   threshold. 
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Figure  8 . 3 


False  alarms  of  the  recursive 
filter  at  frame  2,  using 
two  thresholds. 
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Figure    8.4 


False  alarms  of  the  recursive 
filter  at  frame  10,  using 
one  threshold. 


Figure  8.5   False  alarms  of  the  recursive 
filter  at  frame  10,  using 
two  thresholds. 
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The  results  for  the  hybrid  filter  are  shown  in 
figures  8.6  through  8.9.   Since  this  filter  does  not  use 
recursion  in  time,  its  transient  is  only  in  space.   Simi- 
lar improvement  was  achieved  by  using  two  threholds,  as 
can  be  seen  in  figures  8.7  and  8.9,  compared  against  figures 
8.6  and  8.8.   Observe  that  frames  2  and  10  (figures  8.7 
and  8.9)  have  almost  the  same  number  of  false  alarms,  as 
should  be  expected,  since  there  is  no  transient  in  time. 

Comparing  these  results  we  see  that  the  recursive 
filter  presented  better  performance  than  the  hybrid  filter. 
This  result  is  according  to  the  theoretical  and  experimental 
values  of  tables  8.1  through  8.3,  where  the  recursive  filter 
has  better  performance  for  high  background-to-noise  ratios, 
as  is  the  case  (BNR  =  30  dB) . 
3 .   Background  Suppression 

In  what  follows  we  present  some  examples  of  target 
enhancement  (or  background  suppression) .   Several  parameters 
can  be  varied  to  evaluate  the  estimator-detectors  performance, 
In  these  examples  we  have  used  the  same  kind  of  background, 
but  different  values  of  target-to-background  ratios . 
The  characteristics  of  the  background  are 


a      =1,    pv  =  ph  =  0.93,    p   =  0.95,    BNR  =  30  dB 


The  correlations  in  space  (p  , p.)  have  the  same  order  of 
magnitude  as  those  observed  in  chapter  II  for  several  real- 
life  pictures.   The  background-to-noise  ratio  (BNR)  was 
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Figure  8.6 


False  alarms  of  the  hybrid 
filter  at  frame  2,  using 
one  threshold. 
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Figure    8.7 


False  alarms  of  the  hybrid 
filter  at  frame  2,  using 
two  thresholds. 
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Figure  8.8   False  alarms  of  the  hybrid 
filter  at  frame  10,  using 
one  threshold. 


aaaa 
■aaa 
■  '  ■ 
■ 


a     : 

■  ' 
l   . 
a    - 

■  '    : 

aa- 


aaaa 


■ 
a 


■■■' 


1111 

Hi   i 


■  iniiui 

aaaaa 

aaaa 

ill! 


Figure    8.9 


False  alarms  of  the  hybrid 
filter  at  frame  10,  using 
two  thresholds. 
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chosen  so  high,  because  we  are  interested  here  in  those 
situations  where  the  observation  noise  is  neglectable  when 
compared  with  the  background,  which  is  the  undesirable 
texture  to  be  suppressed,  in  order  o  enhance  the  target. 
The  performance  of  the  filters  will  be  given  by 
the  ratios  defined  in  equations  (8.1)  through  (8.3). 

Example  1 

Filter:   Recursive 

TBRi   =   0  dB 

Threshold   =   0.5 

Figures:   8.10  through  8.15 

In  table  8.4  the  performance  of  the  filter  at 
several  frames  is  shown.   Observe  that  at  frame  1  the  3D- 
recursive  filter  reduces  to  a  2D-recursive  filter  which  does 
not  use  the  correlation  in  time.   Comparing  figure  8.11 
with  8.13  and  3.15  we  can  see  the  great  improvement  resulting 
from  the  exploitation  of  the  correlation  in  time. 

Example  2 

Filter:   Hybrid 

TBRi   =   0  dB 

Threshold  =0.5 

Figures:  8.16  and  8.17 
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Figure    8.10      Original    image    at    frame    1     (TBR.    =    0    dB] 
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Figure  8.11  Frame  1  after  the  recursive  filter 
(TBR   =  12  dB) 
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Figure    8.12      Original    image   at   frame    6    (TBR.    =    0    dB) 
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Figure  8.13   Frame  6  after  the  recursive  filter 


(TBRQ  =17.3  dBl 
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Figure  8.14   Original  image  at  frame  10 
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Figure    8.15 


Frame  10  after  the  recursive  filter 
(TBR0  =17.3  dB) 
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U3UUJ..3'.  L:.L3LLLLLLLN3NUaa«««aaLLLLLLLL3L3LL33UUUL3L3UUULLLLLL33LLLl.LL 
J3L33N  LI.  JLL3UL3UUU33UUL381Ma333LL3LLLLLLU3LUUU3U3UUU3ULUUU3UUN3LLL-  LL 
X  I  LLI.l  11.1LL'_3L3LL3LL3LLL«M3NL3-33L3UULLL3LU33UU3UNUUU3UUUUU3UUU  LULL 
LL'-LLLJ'-33L3  33LL3L3N3LLLL3ias33LL3L3LLL3L3LL333LLL3N3LLLL33LLLLLL3  3L1 
'.  LLL3L3LL3LLLLLL33LLL3LL3aaaLaL3LN3L3LLL3LLLL33L33LLL3L33LL3N3LL~LLLLL 
-  L3UL3L3L3LLLLL333L3LL33LaaN3LL3L3L3333LN3L3LLLL3LLLNL3LLL3LLL33LLL33J 
n3'.L-_'.LLLL3L3333  0L3L3LL3LL3LL3LLLLL3LLLLL3LLLLL3L3L3LLL3333LLL3NL3uJ 
.-JJ-:-L_'.3I.L333L33L3L33LL3aL3LLL33LL3L33L3L3NLL3L3LLL3LUL3LLL3L3L3'_3LJL 
3L33L3LL33333L3LL333LLL3LaLLLLLLLL33LL3LNL3LL33LL33LLLNLLL333LLLLL'_LLL 


Figure    8.16      Frame    6    after   the   hybrid    filter 


(TBRQ  =  15  dB! 


LU-n3L33L3L3v3LnnLLLnL    nnnU   n3UUUnU3nL3\3n3LnnLLLnLLn    L3\3LLJn.nLn3.  Ln 

LLLUNUULUUUUUUUUUUNLLNLLL3LLNLLLNLNNLLLLLSLLLLLLLLLLLLSLL3ULLN3LLLLLLL 

LLnLLnLLLLLLnLLLLLLLLLLLLLLvLLLL-LLLLLnLLLLLLLLLLLLLsLnn.nLLLLLnLLLj.nL 

•  NLLUUUNUUU3UUUU-- ULULLLLLLnLLLL\LLLLLLLnLLLLLLLLLLnnL3LnLn3LLLLLLnLLnL 

3LLLLUUUUU33UUUULLLLLLLLLLLLnnLnLnLL3LLnLLL3LLLLLLLLLLnLLLnnnLLLLnLnLn 

3UUUUUUUULLLUULnULUUUUUU-LnLLnnLnnLLLLLnLLLLUUUU3UUnUUUUUnUUUU3LLLnLLL 

LLLLnL3ULnLnLLLLLLLLLLLLL'LLLLL--LnLLLL3LUUUUUUUUUUUUUUUnLLLLnLLLnLvnL 

3LLLL3LLLLLLnLLLLLLLLLLLLLn3LLLLLLnnLLLL3nLLn3n.ULLLLLnnLLnLLLLLLnLnLLL 

3LLLLLLLLLLLLL3UU3ULLLLLLLLLL-.LLLLLLLLLnLLLLLLLLLnLLLLLLLnLLLnnLnLLnLL 

LL3LLLLLLL3LULLLLLLLLLLU'_LLnLLnLLL3LLLLLLLLnLnLLLLLLLLLLLLnLLnLnLL-LLL 

LnLLLLLLLL    LLLLLL3LULLULLLLLL3LLnLLLLLLLLLLLLLL3LLLLLLLLLLLLL3LnLLLLLL 

LLLL3LnLvLLnLLLLLLLnLLLnLLLLLLLLLLLLLLLLnLUUUUUUnUnLnLnLUn.UnUnULnLLLnL 

LLLLnLL3LLLLLL3L3LLLLLLLLLLLLLLLLLLLLL3U3UUUUULLLLLnLLLLUUnnUUnUUUUUL 

3ULLL3nLn3LULLLLLLLLnL3L3LLLnLLLnLLUUUUUUUUUUUUUULLLLLLnLLLnLLLLLLL^L 

LLL3UUUUUL3LLLLLnLLL3LLLLLLLLLL3LL3LLLLLLLLLLLnLn33LLL-nnnLLLLLLLLLLLLL 

vUULLLLLLLLLLLLLLLLLLLnLLLLLLLLLLLLLL3LL33nLLLL3LUUUULLnLLLLnLLLLLLnLL 

LLLLnL'  L'  LLLLLLLL3U3UNUULLLLLUUNUULLLLLLLLLLLLLL3LLLLLLLLLLLLLNLLLLN.N 

rLLNLN'LLLNLLLLLLLLLLLLLNLLNLLL3LLLLLLLLL3LLLLL3LLLLLLNLLNNNLLLL3LL-NN 

"l  LnLLLLLLLn  v • XLLLLLLLLLLLLLN3LLLLUUUU3UUU3U3UUUNUU3NLLLLLLL.LLLLLLLLLL 

ji"  -LLLL -L    LLLLLLLLLvLLLL3NN3LL33LLLLLaNLLLLLLLLLLLLLLNNLLLNLLLLLLNLULN 

Ji     '  LLLLLL    LLLLLLLNLLUUUULULLLLLLLL3aafl3U3LN3UUUUUULLLNLLLUUUL3LLLUUNU 

"rUL<  L>L    LLLLLLLLLLNLLLL3UUUULLL33aaiaaLLULLLLLNL3ULLLLLLLNLLLLLLLLLU 

"■  L'  '  "LLLL'  LLLNLLLLL3LLLLLLLLLLL33aat«aaaLLNLLLLLLLLLLLLLLLLLLLLLNLLLL 

y  ZrLLLLL3LLLLLLLL3LLLNLLLLLLLNsLLnMaaa3U3LU3UUUUU3UULL3NNl_]ULULLLLL . 

-  "vLLLL3LLLLLL-  ■  LL3LLLNLLLLLLL3NLLnnaaa3NLL3LLLLNLLNNLLLLLLLLLLLLLLLL3 

LL       3L3LL-LLLLLLLNLNLLLLLLLLLLL3LL33na3NLLLLLLNLLLUUUUUULLLLL3LLLLLLNv 

3L-LLLLLLLLLLL3LLLL3LLLLNLLLLLLLLL333LLLL33LLLN3U3UUUULLLLNLLNNLLNLLLL 

LLL    LnLL'  LL'  '  LLLNLLNLLLLNLLLLLLLLLa3LUUULLUUUUUUUUUUUUUUUUUUUaLUvLLLLL 

ULLLLLLLLLLL3LNL3N3NLLLLLLLLLLLLUUa33U3UUU3UUUULLLLNLLLLLLLLNLLLLLvLL3 


3LLLNL    LL-.LLL  vL  -LLLLLLLLLLLULLnLLLLLLLLLn33LnLLLnLLL 

Figure    8.17      Frame   10    after   the   hybrid   filter 

=    ] 
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(TBRQ  =  15  dB) 


TABLE  8.1 


3D-RECURSIVE  FILTER 


FRAME 

TBR.  =  0  dB 

l 

(example  1) 

TBR.  =  -3  dB 

l 

(example  3) 

TBRQ(dB) 

P.G. (dB) 

TBRQ (dB) 

P.G. (dB) 

2 

17.3 

17.3 

13.0 

16.0 

3 

17.3 

17.3 

13.6 

16.6 

4 

17.4 

17.4 

14.8 

17.8 

5 

17.0 

17.0 

15.3 

18.3 

7 

17.2 

1.7.2 

15.8 

18.8 

9 

17.7 

17.7 

16.3 

19.3 

13 

17.9 

17.9 

16.6 

19.6 

17 

17.9 

17.9 

16.4 

19.4 

19 

17.2 

17.2 

16.0 

19.0 

20 

18.8 

18.8 

17.3 

20.3 

AVERAGE  OVER 
FRAMES  6 
THROUGH  20 

17.7 

17.7 

16.3 

19.3 

This  example  is  similar  to  the  previous  one,  but  the 
hybrid  instead  of  the  recursive  filter  is  used.   The  frame 
1  is  not  shown,  since  both  filters  reduce  to  a  2D-recursive 
filter  for  this  frame.   Comparing  the  results  we  can  see 
that  the  recursive  filter  (example  1)  presented  better 
performance  than  the  hybrid  one.   As  expected,  the  hybrid 
filter  had  almost  constant  performance  (15  dB  of  processing 
gain)  for  all  frames,  since  it  has  no  transient  in  time. 
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Example  3 

Filter:   Recursive 


TBI^   =   -3  dB 


Threshold  =   0.35 
Figures:   8.18  through  8.23 

The  filter  performance  for  other  frames  is  shown 
in  table  8.4,  where  the  effect  of  the  transition  in  time 
at  the  first  5  frames  is  clearly  shown. 

Example  4 

Filter:   Hybrid 

TBR.   =   -3  dB 
l 

Threshold   =   0.4 
Figures:   8.24  and  8.25 

This  example  is  similar  to  the  previous  one,  but 
using  the  hybrid  instead  of  the  recursive  filter.   The 
processing  gain  was  around  12.5  dB  for  all  frames.   From 
figures  8.24  and  8.25  we  can  see  that  TBR,  =  -3  dB  is  about 
the  limit  of  detection  for  the  hybrid  filter. 
4 .   Target  Tracking 

In  what  follows  we  present  some  examples  of  target 
detection  and  tracking  from  frame  to  frame.   The  background 
has  the  same  characteristics  as  before.   The  first  five 
frames  are  entirely  processed  in  order  to  acquire  the  target 
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1  33L^  LLL33333^  LNvL3afflUPl.l.33LL3333333^naa3an33UL  U.U3UvN3L33L3T1333t  J 
L3TuvLLL3333L^LL33KLL333333L33L33Bfl3l.33LLL^^ 

i  i!\i  l  LMn3Lv^NvL33333H.L33333333333aBB333333l.N    L  -LLL^-^L^LL^  LLLL!.1.. 
^^>-"s"     :""u-^LLL33333333Bflfl^^ 

^nxnnvsv^U.^nnnLLLLLLnLLLLL3LU'_L33033L3ULL.  :K\^^LLL3LLL3LLL.^ LLL 
.  ULLn^L-  LLvs  NLLLLLLL3333333LLLL3n033333LLvNLLLL3033333LL33T311Jl  I 
i"ii  ni  t  i  LLLLL1  svvNNL33L3LL333333LLLLL33an3333L3LNLLL3303l.333L3303a30J;13 

^?^M3MaM33333U33LL3L33333LULL33n33333L3LLLL3aaaOnn333Daa!raauaOU 

ULLLMaaMnmaaaiM 

U33L33in^IU333nnan3n331LL33aBLLLL33333333LLNLLL33nn333LLLLL333Ja3JJ 
Sinia33aann33333nil33n333LL33nD3LLLL3333333Ll.vvLL33aan33LLLL33333JLL;_ 

iaaaflaHnaaaaaBaBaaaaaaflDaaaanara33LLLLL33333LLs^L33Qa33LLLLLL3LaLLLLL 
iaTHMq[ig3aaaaaBnDaQQaa3QnnnnQaD33t.xLLLL3LLLNN  NUL33n3L\s>«NNsi.LLi.Li.i.L 
nS^EasMBM^ 

.  .•.HaFlil'«MHHnnan3B3333333LLLLL3LL^NNNLLLLLNN    ^^^J-^^LLL^^^LL 

m  i  n3J  ^«aauaana3333333LLLL33LLLsL^LL3LLv  nnllljbllllllll^ 
dja-IlrtlJMiiaSi9aanaaa3333L33LLLLLLNLNNvNNsUL3LNS!N>LLLlJLLLLULUU33L3r. 
iMUH  H..WJ39aaaaHaDanaa33333LL33LLLvLLNLLL3|.l.N  :  ^^^^^^LL; 

laaaa  a«<  jganaaaaanaannn33a33L33DL3LL33L3L333LL  ■  L.%\4£>"£^"^ft  > 
HHH,Ha(nMaaaaaaBaaaflaaBDannn3oao3333333333n3LL  llllll3LL3LLLU3B333lll* 

daH«^aiJBlBaBaaBaaBaBBaaaBnD3anD3333n333333LLL-LLLLLULLULLLL3a3a33J.L 

™naaaBaMBsiaaaflaaB9na33n3333B3L3333333333LLL:  -nLLLL3LL3L3UL3QBI33J^3 

33H«Sa.Uia?aMDMnna33D3a33nD33333333LL3L-.x    -LLLL3LLLLLLL30aaU033L 

33aaaaaalaaaBaaaBaanVnD33nnn3naaDDn3na33L333L3sLLULLLLL'-33333DBaaQa33J 
iSaM9B*aBBBBaBaBnDSna33aoBDaaaDnnaaa33Lt.3LLLNNNLLLsuu33333DBBaaa33^ 

aaa«a3BBB«BflaaBaBBaafinB33BBBBBaanBaBBBB333333LsLULLU3B^ 

a9BaaaBaHiaaaBaaa^aBaBBBBBBBBaaaBaBaan33B33LvLLLLLU3BM^ 
aaaBBBaaBaBWaaaBTOaaaaBaBaaaaBaaaBaBBBBBaaa3i.LU33UL333a3aa3BaBaBBDua 

Figure    8.18      Original    image    at    frame    1    (TBR.^    =    -3    dB! 


sX33  •  333U13L    1^3833038^  3ajLLaa3flL3aaflfl'-33B3LL^LaL001-^3a3a33OaLaLO  03 

33^3-3  3D333J3L333!13333L33a3a3n33LU]3L3a3L3l.aL3aL3333a333333aL3333330333 

333X33313331U3L3L3n33a33na33nn33333a3333a3B3333333333333L3333333033333 

•3L3333L333U333333aa3333L33333a333a333a333333a3a333Laa3a33B3a3a33333L3 

T333U3333L33J3313La333333nL33333LL333B3L3333333a3303OO33a3L3L3333L330L 

H33L33a3333333L333aa3aU3n3n33L3L33a3333a33333L3BeaaaQdUlJUaaa3a333TaTJ 

003311333  3O33333Il33IlL33L33L3333333333a3a3333La33flaa0SaaLJa3L333aa 30  J  JJ 

■pr  Tnn-n333333ng333a333B333Bta3333U33a3a333LBI-3B0gBa3VL3L3033333333a3 

33L3'  U3ai33a3333333L33333Lan3aaL33BB3LL9B3333L:!9338aa«BBT33Ja3:iaaja3:j*J 

OJJP3 03 3aaajOJ 33 a33L3n3333  3333333B333333BBB333BB93nB8aQQ3L33a33'.  333333 

3L'  L3333a23!-33333nL3L3333a3B3B3333333333B33333a3B3aBBBaBaBUUIIBiaJIJllllll  I  II 

itiw  if  n-mmTnm3L33B3333333an333LB333BLaH.L3BB3aaBBB331.3333aLaa3Tm 

3J13J333n.m33l.a3333n3aN333L3an3LL3B333aa333333an33BaaN33B3LLQa333a  133 

1330JJJ'  nmJJJ33Jn3Ja33a3333333333L33383333333BB33aa333L3aa3333333r..33 

aaiaja.TJ33  3J3n333333aja33333333033333auaB333Q333BBL33QLL3a3a33333l.XJJ3 

yiaianiJJ  IJ3fl33nLfl3La333a3BB3B33333L333BaB3333LBQ3fl333LJ3333333J3JI  LJ  '. 

310113:  J3J  iaJ33La33333333n33333L3L3B3333aQL333L33B33BB333L33333D3n3-'n  n 

V  L3  I  J  13  j  1  ljajaJL33LJLJ3a3LaJ3QJ3aa333aoaaa333B3aL3Q333aBL333333aCL.-' 

r  n  mil  Ml     )l  L3*.J33B3fl3  33LL33L33333L33L3B3aaQ33L3LB333L3BaaT33333L33J  J  !  ' 

,     I  1331)1  l     1  min3333\J3a333L33a3B3Q33QLLaLaBfla33333333333SQ3nn3aa3J3J3  : 

!     !  iai!»'   iaJ3aB3333t.33333B33Bl.3L3LflL33BBBaBB3333333U3a3anB3B3Laa33J.133.1 

ill     I  I  1  I  I3BJJ  IJ3JJ33333aa3330333333aaaLaa33LL3LLB3LQ33aBBL333333LJJ  1 

■  an  3  l  i  i  IU  i.TT33nj  iaa33333B333333BBBBBaaNaa3333a33333333aLa33a333L33.IJ 

1 1  ri3i     I  13  im33a333333333a333a3L333B3B333Ba333La333B333B33L3aaaflJJJJT 

i  .  i  1  U3r.l]jai33333nflL33Lja33J3nL33a3naL33B3v3BL33a3333BLa3Ba0n<-33JJ  J  1:1 

11  iJj;i311J3LJ33L3L3Lfl3333333JJ333333a3QL3a333333333333Sja33aa3aa3,-.it 

TJ33T    l3J3JJJ39U333JJ33333aa33aB333333L3L30333333033BB33aa330a33B3BL-:  J 

J  Ut  J  llJJ3J3a9La33J33B3aa33B33Ln33a3L33aaL33B3LaL3333Ba3L330033333Jl 

j-  "-aJTB  nQ,lJ'JdaB93BL33333eaB3BB333333B3B3B33BaBL333333B3aBB3B3aaL33J3  1 

3  13  33JQ  1  1039*39389^  a3333BB333Bn33LB33333333LLB33B3a3Laaaa333La33L33Jl  1 

■  ;  3333  ]!33a33BaaaaBB333aaLn33333BL333afla3333a33L333333a3aaL33L3aQBa3ns-| 

■jj"TJJT n3  33-_a3333QL33BL333LB3L3BB333a333a3a33333a3a3a33aa3033Baa3JJ3J 

TJT-3  3-.JJ3an3  3a^aa9aaBBaBBBa3333Q3333B3333BL3333BU33B333a33L33a0aL3aQ'3 

hj  nu:  in33a3a3aaBBaaaaaaaaaaB3BLaB333BBB3B33333L33L33Laa3BB33B3BLajua 

Figure    8.19      Frame    1   after   the   recursive    filter 
(TBRQ    =    9    dB) 
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^^ssrM3LLL333LLL33333LL3L-LLL"    sCOJ33LLLLLLnnvLL3300JUUUJI II H+H H-HO 

,^^LLL3LLL333LLL333JJLL.JL    ul  lLLLLLn  sNxLL3a333333aOtraEaCT3U0a 

L^"V:LL3L33U^33aMn3333LtL3ULLL30D533L33LLxss 

n-i  v    ii  i^TTn3Hnamn3333333LL333L3L3a0n33L33LLLvLL3a0aaaaOroaBaagaagS 

?^   LLLU33SS3S3333S 

33LLLL3  1M^S^33L33L3333L333DDDa3333LLs.LL33^aa3333^ 

"       t,  TiSa5^an333n3333t.3l.l.LL33LLL3a3a33333LLNNLL3309BDa333naanaiara3 

« mm  iiB^S™22innnn3n3L3LLLL33L3L333333333LLNvLL33Daaa333333ana3a333 

v,m  '^^Raa^Saaanan3n3333aQan3aaDQaa333LLssLL330aan333333anQ3  11 1.1 

'li:l3S-i'l-lfljqaDaHBnonnDa3L33LL333333333333333LN      ^33a333LULL3L3LLLL  . 

■i^S"J3annBBnannnnn333L33aa3onno33^ 

TlM'«-IM=l'H'JJ>JdiaaaHBBaaaan33333nnn3a0DaD3333a3LLNNLLLL3333333L33ann333J. 

i V-!  -in*-, «aaaaaaaaaaanna9HHa33nannannn333333Li.v\vLLLL333333333aaa3333L 

^n^SaaaaaaanalaB^^ 

:;,,„,„  , , ,  ,J,ifiaanami5aaaaaaBBDDBanaQa8anaa333t.LNv-Lt.LL33333333aoaaduuuu  j 

^•iyj^Tirtqa>inaaaaaaaaQBaDaaaaaa$aaanaQ33LLLL'.N  \NNLLL333D333uaaaaauuuj 
^^aaaiflaaaaaaaaaliaaaaaaaaaeaaaaaaaaaLLaLLv  ssxU.L333non3aaaa39aon  3 

SaBaa*aaaaaaaaBa^^ 

qqqallnHMiB9BaaaaaaBwaaar^ 

SaiBaa..aaSS 

33aaB3aaia«aaaaaaaaDa99naaaa8aaaa9aaaaa33333LL^s-^L33aa30333B9Ba33aJ 

333aaaaBa«««aaaaaaaaaa9aaaBaa9aaaaaaaaaa33a33L--^NNL333a3333a9aa333na 
Figure    8.20      Original    image    at    frame    6     (TBI^    =    -3    dBl 


l\L\UU\vL->    L^Mnv^    LL    L-    vnLU.U.LLnn\    nJnlJn    ULnLL33    nLLnLx      nnlU3 
U.nxLsLnnnLnn3LLLnxnnxLL-  nnxLLnnL3LL-  -LLLLL^LL    nLLnLLLnxLxLnnLLnLLLL    \ 
LWxnLnnLxLnLLLLLnnnLLnLLLnxLLLLLnnL    3xLn-xLnnLL3LLxLsLnnnnnn;h.LLLxLn 

nnnnnLnnLLnL-nnxL   L\LLLnx.L-v-LLnv^LnL  ,\LnLnL\nn.I.    L\LnL>nwU.vL\nnn>.u. 
LLL    3^UL    LxLnLLnLnLLnxn^LLLL    LnnnnLnLn    L  .L3--LnLLnxLnnLLLLLLLL-  L-  L--LLL 
nLnnLnnnLnLLLnnLnLLLLnn-    L-  L      L-nLnnnLLx-LLn-  LLLnnLLL-  LL-LL-LLnL-        LLx 
•s\LLs\L'^n:Lw\\M\    LM\^  .\LLNLLvLLLv^LL*_NNl.I-LNM.Nt^vM.U.M.LLN^  -L-LLn 
\LL-    \Lv^L\LnLn\vvv\LnLL^LLLLLLL\L-.\sn\\v\LL\LLLLL^LLLnU.    Lv^O^L-.*  Ln 
nLnnUvLLnLLnnLnLLLLLnnnLLnn-  -.LnnL-Lnx-  •--  nLLLLxnLnnLLLnnLxLnLLLnxLn  L  \L 
3.3nxLn\xLLnx3\nnLLxLLnsLxnL\xLnxn\M.\M.nLLM.xx3n.\LLnLxL3nnnLnLnLLLL    !.x 
t,\ULNNLvv^^NLNNLl.NLUNLNNNLl.vM-LaLLsLvANM.NNNM.LNN^L^l.H.NNNN^V.^N3L^-  '-L 
\nv\^L\L    L-  .nnnLLnnLLxnLLLLLnLnLLxnLLxLL   xLLL    nn.\  .  LLs\l.'  nM.Ln.L3.  ni_Lx> 
L-L-'  L-nLnnLLLLxxLLLLLnLnLLnxnLxLxLLxLL-.Lnxx    LxLxLxnLx3xxL3LxLxnLn--.- 
xLx"  vLLLxLLLnnLnnLnnLnLLxLnxxx3LLLLLLLLxLxLLLxLL3xxLx3x.nnLx    LLLn-LLL--L 
-  L.LLxxnn3LnxLxLLxLLxnxnLxnxLL--xLLxxxLLnLnLLnnxL    LLLLLnI.nnvLLL-.LL-'  -- 
LLL-  LL-I  L-LL-nLLLLLnn-  LLLnn1_nLLLLnL3nLnn  vLnnnxnnnLnxxnxL3LLnLLnLxLLLLL 
L-L-L-  -L     -l-LnLnLnnLnLLnnnLLLn-nsLLnLnLnnLnnLnLLnnLLnnnLnnnnLLLnL  >x\..\ 
■      L--LL --LIn.sLLLnnLn^LnLnLnLnLnLnLnnUnnLL    Ln3nnnLnLLnLLnnnLnnnLnn-     LL 
"j.    ,L.         .1    v.nnnLnn    LLLLnnLnnLLL-nnLnLnLnLLLnnnnLnLLnnLLLnLLnnnLLnLL-.  ■- 
I,  ~,      L     L    "    L-    -LnLLLnLnnUL-.n    nLLnLnLLnn    LLxxxLLnLnLnLnLnxLLNnnLL- LL    XX 
3i      •  3.        ;  !..{    LLL    3n,-LnnnL- LnLLnLLnnn    nLLLnLnLnnnLLnLL\nLnLnnnLLnLn-n-n 
j       j  1    .l.>LLL-L-nnnLnn3LLLnnnLnLLLLnnLnnnnLn\nnLLnLLnnnLnnLn-;.LL 

•I  "LLL  Ll"--LLLLnL-LLLL3nnLnLn  xxLLL-LLLLxxLLLLxxxLLLL . LxxLLxLxxxx v n 
,  ■"  .  1  -L  'L-LLLxL  vLN9a8NLNLLLLNNLLNNNNLNNNLLNNsLLNNL3NNNLNNNNLL-  LL 
".      |_l  LL-  LLNLLLL-aaaaaLNL-  LnnnLnnLLLn3    nnL    nnLnLLL    LLLnLLLn  -  LJ-  \ 

-      LL • L       LLLLLLLLn- NLNaa«««««3NLLNNNNLLLNNNNNLLNLNLLNLNNLNLLNLNNNL3N - L 
L       '      -L    L-L-  1  '  LNLLLLNL3118aaL\NLNLLNNNxNLLNxLNNLNLNNNNLLNLLLL    LLLL      LL 
~    '  -LL-LL     -L     -  .NNXLLLL-LaaaLNNNLLNL-.LLNNLNNN^NLNLNNLLLLNNLNLLLLLL-  LLL' 
1  ;  "■  u    .:.-•_       ,LLLLNN-.NNLLaLLLLLLLLNNNNt.LLNL-LLNLNNNNLL-.NNNLLLNL    L-L-L- 
""'  nnT'  LLLL    LLLnnLLnLL3LnLLLnnn\LLL-nLLnnLnnLnnnnnLLnLnn\LnnL3\LnL 
""lnn'L    LL'LL       'LL    aLLNLNLL-LN-LLNNLLL-NLNNLNLNNaLLNNNNLNNNLLNLLNLLLL 
L-  -nnL'  LL-  ■  -LL-LnnnnLL- LnL- LLn nnL-n- LnL    nLLxLxxLxxxnxLLxLLL : LL s -xLxxLL 
I  n .LL~LLL-     LL- 03     -LnnnnnLLLnLLn    L    LLxLLxLxLxLxxxxxxLxLLxxLxxxxLL    LL    L 
L^-Ln-LLLLLL-nnL    L3n    Lnn-.nvLLL-nnLLLnLnLLLL'NLL    nnLLLnnLLnnLLnnLnnL- 

Figure   8.21      Frame    6    after   the   recursive   filter 
(TBRQ  =15.8  dB) 
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i  i  t  i  si  Jiaaaaaaai  333333331  I  31  »i  mmmm.i  mi.i.i.mui.Tmf)Tnmyw 

^,-n.i  1 3jaaaaaaaa3aa333«  331,331/  »xlllulu.ul>  \m  i  i  >.m  i  3333303030003330033 

I  i.i.i  L33aaa3iaaaa33a333l  t'T""-'t  mmi.isi.M.iULi.iim-»nmmiiTnngn 
w.  ■  ■.  vl33333B33H.333lLH.M3t.H.33L3333LH.L\L>  LLLLLL3330aOOOBaPOOOOOOQBB 
\.  N'.LLL3LLL333LL333LnLLnLUL>.LLLnLI.LLH.nv  vsvvv„sLL:i:Jatraa:i033:ra33  3;ia3a 
v.v\vvLLLLL'*L3L3LLJLJLnLLL33LLL33LLLLLLLnn  L^LLLLLLJJnBaaBaBaoaOQUUUUUO 
\NvN\L1.33LLL333L33333LLI.L3LLH.3L'>LNNM.L'.\,«.L^\L\Lt.L33aaaa0a8300aB0aBaBB 
vLvsvsLJ3LL3IUIU33333LL3333LL3a333LLLLLLLM.LsLvvLv33asa3aaa3UCraorcn3aEJL) 
LLvsvvL3333IUlIlBJUlinil3333333L33n33333333vLvLLLLvLNvL3DBairana3TJCHBBrcEOU 

LLN\\LLJ333iiaaaaHiinana3J333i.LL3aLLLLLLLN\\LLvLNLxvaDaaaaHonoaajufcJUuujj 

LL        M  1  Ti-nwinmnm-mi  11  LL3ra33333LLsLsLlsLLLLN3308<aa»aaBaaBBBBBBBB 

ln  •     vLL3333nnnaHnma333L3LL33nta333333L3i.33LLLLLLL3aBaaBoooaaBBBoaaaa 

LLvv  ^LLg3331iannHanan33333333aaa33333L333L3LH.l.I.LL3309aiMBBaaaBBBBBOaa 

LL^v-^LLJiiiiniiBaaaannannnn3ii3333aaDnna333333D3t.LL3UL33DaaaaBana8BHQauaaQ 
L3vNLLJ3HHaBaaHaaaaHaunnann33aaaaaa3]3333LaLNL\LLLJ3Da9aaDnoaBeaa  aaiai 
.3LL3333aBaaaaaaaaaaaaaBaaaaaa3aaaaaa333333LNN  ^LL33aaaae€B3aBanaai_rj  irn 
.3LLLL:niinaaaaaaaaaBaaaanaBaaaaaaaaa3L3L3L3N  wL^Baaaaaaosesaaanai 
lai Jiu:i:iHHaaaaaa«aaaa9aaQBaaaaaa8aaaa333cannNL\LL3anDaQaaaa3]3a33LLL3L 
xu."]jjaamaaaaaaaaanannnaana3aHaoanajiiLL333L^N  vvi_L33naaaati333a:]3Li.LLL 
la. j j j in  ia.uda3aaaQaQQDaa3aa30aaaaaaa33CL333L\LNNsuL33nnnaaa3aosaa333a3 
3i33!i3H.i:idanaaaaaaaaaaaa3DaD3aoaaaaa3333333LLLNH.LLL3aaDa3n333na33LLJL 
^i33aQaad33aBBaaaaaaaDan3nanaDaaa9aaaa3±i33LNLNLNLLLL333a3a333naa3LL3L 
aaajaa33'jQananaaaaaaanaDnaapaaaaaaaana33L33LLLsLSNLLL3aanDn333ana3LLL- 
jaa-.n:iri.naHoaaaHMbaaaD9naaanaQBaaQa[]33U3L33L,-L  LLL3L3naanaenDQaaaa  jl  il 
aaaaanajnnanaanaaaBaaaannDBaBqaanaaan3d3333LH.M.Li.LL33aaaoaaaDaaaD3iJL 
aBa93aaaaBaaaaaaaaaQBan333nn3aasaao333LLLL3LLL: NssLsL333aaaannaaatm3L 
3aaaaaajaHaaaaaaaanaana3nanDnaBaaaana33i.LLLLN^  vvsi_333nnn33naaaD033L 
aaaaaaaaiaaaaaaaananaaaDDaaaanaaaaaaDaLLLLLNNs • •  ■ ss\i_33naa33nBaaa333L 
aaaaaaaaaaaa8BnaaaanaQannaaanaaaaaaana33LL3i.ULvvN\\M.nonBBaBaBaaanD333 
a933aaaa«aaaaaBaaaanacaaDaaangaiBaaaaQ33L33LLL~  -••NN',i.3Daaaaaaaaaaaoao  i 
33aaaaHaaaaaaaaaaaaoaaaaDaaaaaaaaaaaDa33i.33LLLN\\sNNLanaaaaa[]aaaa33aa3 
3"j3;jB3BrtMnaao3Daaaaaaaaa9aauaaHBaaaa3  33LL33LLN  --'.l^  L3caaaBBBaaaau3  3cn 
an3aBaa8aaaBfflanaaaaDaQaoaBaaaaaaaaa3333t.t.LLLN  :  ■  v3QDnua33i]ana3L33'j 
3anaaaaaiMaaaaaaaaaaaaaaaaaaaaaaaaa333tLUi.Lvv  NL33n30333nnm.uL3L 

Figure   8.22      Original   image   at   frame    10 
(TBR±   =    -3    dB) 


33LLLvvl.vvt_NLNLLLs    LnnLLL    LnnLL3v    LL :  on  n3nLn33LL  ^  L\\'     ss    LssvLv    vLLLL 
HLvLLvvLLLv    LLaLnvvLvLL'  swsvL-  ^\L^  n[_nsnwLLvL-  LL LvvLLvvsssLLLsLLsLL 
'     vLvLLLsnvLnLLnLLnn  -ss^M.LnsLs-.-.|_L,nnlLLn-,L^nLnsLLnnnLL    LLLLwLnLnLL 
LnLLL-LL-  LL-  LL    LLLLL-  >L\>^  ^LNLLCN^t-LNSLvL^LN^  v.U.\vU.Ll.vLvv.U.v.Lv.vvL 
vLLvsLnvvlLL-LLnvLLvL"  -L^LL  -LLL-L--L'  L\'-^LLv^L^LLx'vL-A\\\^sNLa-  vL'-L 
LnLsL^LL^  -L^-nLLLLLLnLnLLLL    L3L-  LLLsnssssLLLnnvsLLLsn  --LLvLLnns  v^LLxLL^  L 
"  nnvLLnLLLLnLLn\vnI_LLnns(_l-  LL'     Ln  \ww>.LLLLv    L  ^LLsvL-  vsLLs \\\3L  -  •  v *•-  L 
LLnLn!_LI_[_ss[_[_sssssi_sLLnLLnn  ■  LLLL'-  ~-s\\LL\L\\L\Lv,L\\L\\^\  vsLvLLLsvL  -  L  ■  •- 
LLvLLwln;_sI_sLLLLLsLssnsssLnL    L    sLLnLnsnUnwUnLL    LLLvvvvLL-LvvLnLL    -     L 
sLLLLLvss~  nssI_LLvvLLvvvvLvM.wLn    LLLnnLn    nLnvLLL  ^LLLnLLLLLnLLLLLL n^  •  L  ■ 
-n  ■»[_>. [_l_\[_xI_LLn nsLnwLLvL vvLnssu_n Lsvvs-  LLssLLvsLs  sI_n  LnLLnLL- LnLM.    LL 
v"  LL'   >L    L^L  *'-Ls  -LvvLLLvvnsi.nsi_LL-  -LvvvLLvv-Lv  nvvLnLnnssLLnnnLLv-  L      -  ■  y  I 
X>L'N^>L>  --ns-lM-LLnnLLLn  -L^-  LnLLnLLLLLLLnnLLnsLLLLLLLsn  3LnLL-  nLLnL 
M^nLLvL1.  LnLvnLLnL^nnsL.LLnMLLLLnnLnLLLnLnLnssLsvLnv1.v(.vLssI_sI.LLLvLLn 
-.(.-•  sL    LL    LL-  nLLLLnLL^-  LL-  L    -  L  nn  >  LLnnn  Ln1.ssnsLLs3nLLvsssLLnLnns  w\  s  Ll    L 
LL  -LLw-.svs    3L    n{_LLnLLLnLLLLnLnnLnLLnnsI_nsLsx    LLnssssI_sLLLLns;1ssU.nL  "^ 
'■■l.'     L---L    --L     -  ,M.vLv-1_LnnLLLsLLLsI_Lvs\Lvs    L'  vl_ss  W  nLLLsvssLLLnLn  w   ■  Ln  . 
--   •  -L  •  •  L--  nLLLvn  .LLwLvLvvLv    LLnnssLLLLLsn[.nvssvLLvvLLvssvvsLL    "LL 
>:.    ■_••  ^L  >L  -LL'  L-Nl_LLNNs[_vvi_avNNLLNNLNLLNNLNLNt.NvLLLvLNLN\LN  >.        1 
..     -    L  LLL-   -'  LvLLLs  vnLLLn  -LL-'aaavLvLvLvvvLvvLvvvvLLLL    LLn\LnL\L    3L 

L     Li.       LL  Lx"•L\'■^LLL^\^L■•aaiaalL\\->L-•\L^s\^^l^^^\^^\\L^\L^^L'.u^ 

LLLL'-LNLLNLLNNsaaiaBaaLNNsssLLLLNNLNNsLN(_ssLNLNL--LLL    [.'_ 
>L-  v  ^LLLNLLLNssssLaMHaLsNssLLLs    LvLnsLLnLLLLLn    \L\\M  i, 
\\  .\\s\\\L[.LvL^xLaaBv  nsvLLLLLLnLLLnLnLnvLvLnLnnLLLL^  '      J 
L-  s-.nsI_--LnssLLLLLL    aNNsLNssLsLLvssl_vvLvvvvLvvsLLL-L    SL    L 
LL     _  LL    1     ■     LLLLLLnLnsL -s-   • sLsssLnLnLLLnsLnvLnnLnsnvLvLvsLL.nL'  LL 

LL    '..       LL    LL       L      L-ns[.nL-L    '-LL3n--LLLnLLLnnLnsssssLLsL-.sI.LnLnv\sLL    L     - 
L    LL    LLLL'LLLv  LvvLLws  ^LL^Lx^LLL^^L\^^.\\^\^^LL\\^LL^^\l.\LL^L 

L-  L_L  ■     ■  LL-L3LL  ,^  s(_LvsLnLLnnsLLLn  nLLnLnLLnL    LLLLnLLnL^  LLn  sLns-  v  -3 
v    L    LL'   -  L  •  ■    .-■  \  vLssLLLnsI_LnnssLLLsLLLssL\ssnlnnI_LnnsLnnnLLnn~  nLn-  ■  L'  L    l 
-■'  nLL-   >    L  •LLL^LLN^LLLLLL^^^     nLLLLLnn[_n  LnsLLnvsLLn  ■  nCLLLLnLnsLnss(_n  ■   xv 
L^v.L'-''LL'    v"  LvLLLLLvsLvLvLvLLL-  nLnLnvLnM.sLLLLLLLLnnLnnLLI.vL  o  LL"  L" 
L3vnssv,LL    L~  -- nvLLvLnLnLn  vn(.ssi.nssLnLsssvLLLLvs  xUsLns(.nsvsvlvsvLL    LL - 
L     -.sssLLss-  LsLLLLLsLLvsLssULnvLv  vL3vLvnssLnLvnUnsLLLnLnvLLLLnn1.nn1.n[_    ^ 

Figure  8.23   Frame  10  after  the  recursive  filter 
(TBRQ  =15.9  dB) 
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I     1 

L 
L 

L 

\_ 

L 

LXJlsaaaaULl.  3U3UUU3^U  n  "^  Laa3aL-LL3L^3L3^■^*^O^L33^3I^.aL3>  L3UQ-OU 
N3LUL33L3L3LL33LLLLLLL33L3LLLLLLL3L-  3LLNL- LLLLNLUUULL3LLLLLLLL333L03U3 
ULL3L3L3LLLLLLLLL33LL3LLL3LLL3LLLLLLLLLvL33LLLL3LL3LUU3UnLLLLLLL33L3LL 
LLLL3LLLLt.3L3UUU^33U33U3UL3LLLN3L3ULL3LLLLLLLLLLLLLL333LL033L33LL33L3 
X_33LL3LLL333L33LLLL3LLLL3LLLLLLLLLL33L-LLLLLLL3LLLLa3LL3L--wU3ULv3L3L 
1333LUL33LL33L3LLL33L3L3LLL3L3^3"LL3vL3nL\LLLLLL3LLLLLvLLLLLL33LLLL3-)3 
L33LL33LLLLLLL3L3LLL3L3L3LLLLLLLLLLnL3U33UnLL3LLLLLLnLLLLLL3LLL33LLL-3 
1LLLL3LLLLLLL3L3UUU3U3U33UsBULLL3LxLLL33L'.LLLLLLL33nLL\LLLL33LU3U3LLLL 
3a^LL33LL3L33L33U333UU3ULLL3LLL3LLL3L3LNLNLLLLs3LLL33LLLUNUU0UU33UaUL3 
3L3LL3LLL33L3L3LLL3vL3LL3LLLLLLLL3LUULLLLLLLLvL33LL3vLL3LLLLLLLLLLLJL3 
3LL3U3LLL3LL3L33U3nU>.LLLLLL3L3LLsLL3LLLLnLLLLLLLLL3L33U3UUUU33U333ULL3 
3LLL3LL3L33LL33333Ln3LLL3LLL3LU3U\L\LLLLLLL<>LLL3LLLLLLL3L\LLL3LLL3LLLL 
LL3LLLLL33LL3LLn3LL3LL3NLLLLLL3LLLLLLLN3NLLL3LL3LULLLLLLL3LNL33LUU3U3U 
XLU333L3U3ULLLL3L3LLL3UU33>3U3U3L3L\U3ULLL3LULL3LLLL3L3UULUU3U3U3ULL3L 
3333L3UL3L3LL333LLLL3L33LLLLLLL3L3L3LL3ULLLL3LLLL33LL3LLUU3L3U3ULL33LL 
LL3LL3LL3LL33LLLn3LnL3LL33L3U3ULLLLnLLL333v3LLL33U.LLLLL3UU3U3LL3LLv33 
3L33LnUULLLL33L3UU3L3LUL33U333U3ULLLLLL33LUL3nL3L3L3UUULLL3U3U3UULL3Lv 
;H3LJlLLJLLLLLL3L3L3LL33LLx3N3333L33LaL3333ULL3333LNa3U.L3LLLLL3L333LLN 
-13LL3333L3LLLL3LU3333LLLLLLL3L3UUL33333L3LLLLL3\3L3LULL333L33LL3LLLLL 
31LL33BLJLL333L33L3L3L333LL3L3333L3U3UU3U33UU3LL3LLLL3N33LU33U333LLL3L 
3JL33t.3LaLLL3LL3L3NUULL3L33333LLL3L33naaaan33U33U-L33LLL3LULL3nL3NLLL3 
33L3LL33L33L33333L33U33U333333333n3333U33UUNULL3LU3vU333333\UUUU3ULLLL 
LL3L3L3L33LL33L33L3333L333B3333333nnBaan333N33L333LL3v333L3LL3LLLLNLLL 
33LIlJL3333L3L3L33333LUL33aM33LUL3Baaaan33L333L333U3L3L33UL3nLL3LLLLLx 
vLL3L333L3333LLLLLa3LL3LaMM33L3BaBBnnn^LL3333333LLL33LLL33LU3ULLLLL3 
L3LL3L3L3L33L333333L3L3SBBaaBnL3LLa3UL3N3L333UU3UaUJ33U33LLL333L3LLLLL 
33L33ULL3LL33L33L3333ULLaBBaani]naann33LLLL3LLLL333LLL33U.LL333L3LLLvLL 
3L33L3L333LLL333LL3LL3LLNaaan3aD\3n-OLLLLL3LL3aLL3LL3LU33LLLL3LLLLLLLL 
3LLL3L3L33L333L3a33N3L3LLaaaan3L'_3L33LL3L333333L33^33LL3^33L3LLL3N3L3 
3L3L3LaLL3LLLL3La33333333aaiaaaa93nL3L3t.333LL33333U-L3U33L33L3ULH.L3LL 
LL3LL3L33aLLLLL33a33LLn3Laaaana3L\333333N3L3LLLL3UJ--L333LL33*.LaaaLL333 
a33LL3LLL3L3L3a33N3L33B3Laaan3N33LL3333LLLL33L3LL3Ur-33LL33-3L33LL3L3 
L33LLLL33L3333L33L3LJ3LL3anaL^L33LL33a3L3,.3LLL3L3L'.-J'J.33U3L.-;L33a*-^L3L 
aL33U3LL33333U3UL33n3LL33a3LL3L3L-33LL3U'-J3LLr3LL33LZ'.',.LL3nZ3nL--L'-3LL 

Figure    8.24      Frame    6    after   the   hybrid   filter 
(TBRq  =12.5  dB) 


LLL\3333L3L3\3LL^LLL\L    w*L      33L_nL3\L3^ 3n3LxsLLLn3Uv    L3n3LLJnsUs3    Lv 

LLLLnLLLLLLLLLLLLLnLL~-LLL3LLLLLL-L.nLLLLL-  LLL3LLLLLLLLLLL3ULLX3LLLLLLL 
L^^UL-vLLLLLL- LLLLLLLLLLLLLLLLLLL- ' LLLLLLLLLLLLLL3L3LL^ LnnnLLLLLLLLLL^ L 
^LLLLLvLLL3LLLLnLLLLLLLLLnLLLLvLLLLLL3--LLLLLLLLLL\LL3LnUn3ULLLLLLLLM- 
3LL3LLLLL333L3LLLLLLLLLLLLLLnLLLLvLL33LsLLL3LLLL3L3LL3,'LLLLn^LLLL^L^L^ 
.IUL3LLLLLLwLLLLLLLLLLLLLL3  LLLLL'LLLLLL' LLLLLLLL3LL>LLLLLnLLLL3LLLLLLL 
LLLLvL3LLvLLLLLLLLLLLLLLL-LLLLL^L'..LLLL3LLLL::LLLLLLLLLLLvLLLLvLLLa^  L 
3LLLL3LLLLLLnLLLLL3LLL3LLL^33LLLLL>-LLL33nLLn3n3LLLL'.v^LLnLLLLLL- LLLLL 
3LLLL33LL33LLL33L3L3LLLLLL'.a3L3LLL-L3LL^LLLLLLLLLxLLLLLLLNLLL  --LnLLLLL 
LL3L,_'-LLLL3LLLLLLLLLLLLLLLL3^LLLLLJLLLLLLLLnLnLLLLLLLLLLLLsLLLLvLL^3LL 
LLLLLLLLLLLLLLLLL3LLLLLLLLLLL3LLvLLLLLLLLLLLLLL3LLLLL3LLLLLL.3L-LI.LLLL 
LLLL3L-LLLL-LLLLLLLnLLLvLLLLLLLLLLL_LLLLnLLLLLLLLLnLnLLLLnL-'.LLLvLLL~L 

LL3LnLL3LLLLLL3L3LLLLLLLLLLL3LLLLLLLLL3U3LL3LL3LLLLnL-.LLLLs' -L^LLLLLL 
3LLLL3  LL3LLLLLLLL3LvL333LLL^LL3LLLLLLL3LLLLLLLLLLLLLLL  sLLLLL-3LLLL  >  !.L 
LLL3LLLL3L3LLLLLvLLL3ULLLLLLLLL3LLJLLLLLLLLLLL\Ln33LLL  •  -nUUL'.l.LLLLL^LL 
•LLLLLL;.LLLLLLLLLLLL3LnULLLLL3LL3LLLL3LL33LLLLL3LLLLLLLLLLL-^LLLLLL-LL 
L!  LLL  •-LLLLLLLL3LL3L3U^ULLLLLLLLnLLLLLLLLLLLLLLLL3LLLLLLLLL_LLLnLLLL-  > 
;.I.Lv3vlLLL~LLLLLLLLLLLLL^LL\LLL3LLL_LL3LL3L3LLL3ULLLLLxL3UnLLLLL3LL^  •  -* 

1  IvLL-LLLl.  ~-  LLLLLLLLLLLLLLOLUULLLLL3LLL3L3ULLnUU3LLLLL3UULLLLLLLLLL 
1!  ■_L3Li.  L  ■LLLLLLLLLNLLLL3NN3LL33L3L3LaULLLLLLLLLLLLLLvvLLLLLLL3L3^LLLL 
1'.  . '  LLL'-LL  LLLLLLLvLLLLLLLLLLLLLLLLaaa93L3L\3LLLLLLLLLLLLLLL3L3LLLLLLL 
^'-  '_:  './.  L    LLLLLLLLLLLLLLLLL3LLLL3LL33ai9anLLLLLLLLvL3LLLLLLLLLLLLLLLLLL'. 

-•--^.-_L3LLLL-.LLLLL3ULLLLLLLLLL33aa««aaaLLxLL33LLLLLLLLLLLLLLLLL-LLl.L 
1LL.T ...Li .;-33LLLLLLL3LLLLLL3LLLL^  L3aaaaan3L33L3L3ULL3LLLL3N\L3LLLLLLLL  ■• 
s-  3LL'_:iL3l.LLL-  -LL3LLLvLLLLLLL3LL3anaaa3LLL3LLLLLLLsNLLLLLLLLLLLLLLLL3 
LL  ^3L3LLL  LLLLLLLLLLLLLLL3LLLL3LLannB3LLLLLLLNLL3LLLLLLLLLLL3LLL3LLL - 
3L-LLLLLLLLLLL3LLLL3LLL3^LLLLL3LL333a3aLL33LLLN3L3LLLLLLLL\LLLvLLLLLLL 
LLLLL  LLLLLLLLLLNLL-LLLLL3ULLLLLLLnaan3LLLLLLLLLLLLLLLLLLLLLL3LL-LLLL3 
LLLLLLLLLLLLJL-L3N3vLLLLLLLLLLLLL3n,3na33LL3LLLLLLLLNLLLLLLLLLLLLLL  LL3 
>LLLLL3    L3LLLLLLLLLLLLLLLLLLL33LL-aneaaLLLL-L33L33LLLLLL3NLLLLLLNLLLL 

L^LLLLLL3LLLLLL33LLLL3LLL--LLLLLLLa:39na3LLL:]LLLLLLLLNLLLLLLL--L:!LLLL 
3LLLLLLL  •  LLLLLLLL  -LL\L3LLLLL3LLLLL3I2aanLLLL  -LLLLLL3LLLLL.LL33--  LL3V  L 
■-:.L-LLnLL_LL_LLLLLLLL,.LLLLLLLL3LL3LL33n3LLLLLLLLLLLLLLLLLLLLLLL3-LLLL 
3LwL^L-LL    3LL-.LvLLLLLLLLLL3LLLn3LL.LLL3LL3^*.LLLLnLLLLL    L3nL3LLLnLL- ULL 


Figure    8.25      Frame    10    after   the   hybrid   filter 
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(TBRQ  =12.5  dB; 


(search  mode) .   The  window  mode  is  implemented  at  frame  6 
and,  thereafter,  the  pictures  are  processed  only  in  a 
small  20  x  20  window  centered  at  the  target  predicted 
position.   In  the  examples  we  have  created  binary  pictures 
to  emphasize  the  performance  of  the  filters. 


Example  5 

Filter:   Recursive 

TBR.  =  0  dB 

Threshold  =0.5 

Figures:   8.26  through  8.32 

Example  6 

Filter:   Hybrid 

TBR.  =  O'dB 

l 

Threshold  =0.5 

Figures  8.33  through  8.36 

Observe  that  the  performance  of  the  hybrid  filter 
is  inferior  to  that  of  the  recursive  filter  in  example  5 . 
It  can  also  be  seen,  in  figure  8.36,  that  the  tracking  is 
better  after  the  first  5  frames.   The  reason  is  that  in  the 
window  mode  the  measurement  error  of  the  centroid  position 
is  limited  by  the  size  of  the  window. 
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,.;..LLNL3333333LLL3-jn33L3LL333333333333aaB33naaa3333L3L3L3a30330333L3LL3 

LLLLsL3333333LLL33333LLLL3333333L33LL3Ba3L333a3LLLLLLLL3333LL3333LJLL3 
vsw  vLL3LLLLv»  LLL3LLL- vN333333333LLL33B3L333n3333LLLLL3333LL3333L3333 
LLLLLLL^nLLLLL^n-  33L3333333LL3Ba3L3aBa3333LLsLL3a33L3:]OOaoaaOa 
\ssSN\ua3LLJl.LLLLL3ULsNNN33LLL33LLLL33nnLL3a3n33333LL3LaDna330aQOOaOOa 
LLLL--L33I0333LLL3L33LLLLL3B33333LLLL33a3LLL33a33333LL3:)e8TO3TJe_BQaCO3a 
LLLLNL31IU3J3LLLLL33LLL\L33L3LLL-  NNNL333LLL33a33333330nBaaaOafWaaaaOOa 
1-11  t  l  -|HTiTinTiTin-i-|-iTH  nmi  I  I  I  -mil  I  i  >\>  sLL33LLL3LaLL333L3a3aBauaaUfcUtJUU33a 
33i.LL3Hail33mi333a3aa3ULLL3LL3LLLv\L\LL33LLLLL3LLLLLL3a3an333aaSQSQC333a 
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Figure  8.2  6   Original  image  at  frame  6 
(TBR±  =  0  dB) 
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Figure  8.27   Frame  6  after  the  recursive  filter 
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Figure  8.28 


Original  image  at  frame 
(TBRi  =  0  dB) 
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Figure  8.29   Frame  12  after  the  recursive  filter 
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Figure    8.30      Original    image    at   frame    18 


(TBRi    =    0    dB] 


33nLsu3aimaHaHanaiiajianBaaa333LL333333BaBa 

i  ii  i  si  i  -i-inriFmTiTTiTinn-i-i-i-ii  3333331  mi  r  h  mm 

LLsn    ^sLL333333LLL3LLLLLL333333B3333LJBn3 

•■Lw  .  NsLLLL3LLLLLLLLLLLLL333333B3333LLaBL 

'.3LLLL\LLLL33333333LLLL  .13LLLLL3LLLLLL33L 

L3LL^vLLL33333L3L3t_LL3LJa33LLL3L.LLL.LL33l. 

LL^LLLL3333Ba3333L3LLLLLL3LLL^LLLLL3333BL 

1  -ii  i  i,-ii  -n-itinmnnn-iTm-i-n-in-mi.L--ii.Li.Lii.-inL 

L3LLLL^L333naflnaiinaiinaa33a33LLLLLL^l.LL33L 

L3Ll-LLL3333aa333B3naa3333BLLLsI.LsNNNLL33L 

LLLLH.LL3333n3333L333LLL33LL.NssLLM-LI.L33l. 

LLvnsLsLLL33333LLLLLLLLLL3\snssLLnLL33303 

L3LLnL-s3LL33LLLLnLvL"-LnLLns      sssssLL333L 

33LLVL -  - LLL3LLLLLnLLLLLLLLn 

33LLLLs-n3L33333LLLLLLLLLLLw 

tLL^    n       LLL3L3LULLLLLssN^Lss 

LLl.sws    LLL33LLLLNLM.NSNM. 

33  11-sLL    333B3333JL3L3sLLLLn  ' 

VVI1L3LI    mUa333LLLLLL"-LsLs 

1I1.33L33  nJ3333333L3LLLLLLLs 

iii03aaujdaauaaaaa33L3LLLLLN 

33  3JLJ.)'   n33aaBBD33t.LLLLLNN 


sLs 


\LLL3L 

\LLL33 

s    !    sLLLLL 

■       \nsLLL 

: sssLLsLL3 

ss      st.ssl.3 

nLnn^LLnL33 


m 
a 


aaaa  i  iss 
aooQCHun 
□nan3Q3i 

D33333L3 
0DQ3a3L3 
0T13333LL 

□annn3L3 

333333LL 
33333 TL J 
LL333LLL 
LL3333LL 
3333JLLL 
33033333 


ssLss    LLLL3B    !     • 

!\\\! , ss\L33nn333L33a3LL3L33B3L3L3333JLL3 
1JJJJJL'_3LJ3aa3aaaBL333333l.NsNLLss  LssL33BB333U.33n3LJLL33L\ LL333 J!  Li- 
lt I.I.LLL  LLL3333aa33L333333LLNLL33LN3LLL3BaaBB3L333n333I.L33LL33nnaQ3J-!3 
Lu  I.L__L33LL3333a33L333LL3LN\sLL3LL3LL3BaBaaaBL33Ba33a333a3333ana33 3 :  ) 
'_•_'.  •.LL'-.33333333333LLL3333LNSNLL3LL3LL3aaBaBBB33BaB33a333B3333a33 J  JLLL 
LL.LJJ3l.-J3LLLLL33LL>LLL333LLLL3333333L3a3BaaaaL3BaB3Ba333B3333naQa.!LL'. 
LJ333U3  3n33333333L3^LL333B3333BBn3333L3a3aaBBB3aaaaanaaBaB3333a3a33L- 
333an33anaaa33333L3^LLL3L333B3aB33333t.3B3BaBaaBaaBanoaBaBa33B3D333:i. 

L33J  iaauaaD3B33L3LLvL- LLLa3aaBaaB333333a3aBaaaaBaaaaaaaaoQ33333333L  - 

33B3aaaaaaQaaB33333LLL333BaaoaaBB333LL3a3aaa33333aa33a33a3t.3333333L 

n3aaH33B3aaa3333L3LLL333BBBa8aaaa333L3a3aanaa3anaaansaBaa3333n3J3L- 
^3333HHai3aBBaaQ33L3LL33BaaaBBaaaan3a333a3aaaa333aaa33aa3an33333333L 

L33CnBa8a38BBBB3333L33aBBBBBaaaaBBBBB3303aaBDB333BB3333333L'_LL3333L   * 


Figure  8.31  Frame  18  after  the  recursive  filter 
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Figure  8.32   Target  tracking  with  the 

recursive  filter  (example  5) 
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Figure  8.33   Frame  6  after  the  hybrid  filter 
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Figure    8.34      Frame   12    after   the   hybrid   filter 
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Figure  8.3  5   Frame  18  after  the  hybrid  filter 
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Figure  8.3  6   Target  tracking  with  the 
hybrid  filter  (example  6) 
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Example  7 

Filter:   Recursive 


TBR.  =  -3  dB 

i 


Threshold  =  0.4 

Figures:   8.37  through  8.4  3 

Example  8 

Filter:   Hybrid 

TBR.  =  -3  dB 

Threshold  =  0.4 

Figures:   8.4  4  through  8.47 

Comparing  examples  7  and  8  we  see  that  both  filters 
were  able  to  track  the  target,  but  the  recursive  filter  is 
clearly  superior  to  the  hybrid  one. 
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Figure  8.37   Original  image  at  frame  6  (TBR.  -  -3  dB) 
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Figure    8.38      Frame    6    after   the   recursive   filter 
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Figure  8.39   Original  image  at  frame  12 
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Figure  8.40   Frame  12  after  the  recursive  filter 
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Figure    8.41      Original   image   at   frame   1! 


(TBR.    =    -3    dB) 
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Figure  8.42   Frame  18  after  the  recursive  filter 
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Figure  8.43   Target  tracking  with  the 

recursive  filter  (example  7) 
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Figure  8.44   Frame  6  after  the  hybrid  filter 
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Figure  8.45   Frame  12  after  the  hybrid  filter 
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Figure  8.46   Frame  18  after  the  hybrid  filter 
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Figure  8.47   Target  tracking  with  the 
hybrid  filter  (example  8) 
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IX.   CONCLUSIONS 

A.   SUMMARY  OF  RESULTS 

The  experiments  with  several  real  life  pictures  allow 
us  to  draw  two  important  conclusions  concerning  image 
modeling.   First,  the  hypothesis  of  an  autocorrelation 
function  with  separable  kernels  in  the  vertical  and  hori- 
zontal directions  worked  quite  well.   This  conclusion  is 
very  important,  because  the  derivation  of  the  dynamic  model 
is  substantially  simplified  by  avoiding  the  difficult 
problem  of  two-dimensional  spectral  factorization.   Second, 
the  first-order  model,  used  by  most  researchers,  is  a  poor 
approximation  for  pictures  with  few  details,  although  it  is 
a  very  good  model  for  pictures  with  many  details.   The  auto- 
correlation function,  suggested  in  this  research,  seems  to 
be  a  good  choice,  because  it  includes  the  first  and  second 
order  models,  as  particular  cases,  and  also  permits  a  simple 
method  of  parameter  identification  which  can  be  easily 
implemented . 

The  performance  evaluation  of  the  sub-optimum  two- 
dimensional  recursive  filters  of  Habibi ' s  type  [3-5]  allow 
us  to  draw  some  interesting  conclusions.   Although  sub- 
optimum,  these  filters  are  not  so  far  from  optimality.   In 
the  worst  case,  the  error  variance  was  around  15%  greater 
than  the  one  obtained  by  the  optimum  interpolator. 
Rosenf eld's  filter  [3]  is  the  best  and  presented  degradation 
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around  8%.   Shachar ' s  filter  [5]  has  the  simplest  algorithm 
for  gain  calculation  which  uses  a  very  good  approximation, 
since  its  performance  was  quite  close  to  that  of  a  similar 
filter  suggested  here  which  calculates  the  gains  without 
approximation • 

The  reason  why  Rosenfeld's  filter  [3]  is  the  best  is 
because  the  estimation  error  is  orthogonal  to  the  obser- 
vation of  the  pixel  under  estimation,  although  this  does 
not  happen  for  the  rest  of  the  data  set.   This  conclusion 
leads  directly  to  the  hybrid  filters  introduced  in  this 
research.   From  the  results  shown  in  Table  5.1,  it  can  be 
seen  that  the  non-recursive  filter  [6],  suggested  by  Bar- 
Yehoshua  [6],  although  using  only  9  observations,  presents 
better  performance  than  the  optimum  (in  the  sense  of 
recursive  filtering)  interpolator,  for  correlations  as  high 
as  0.94.   The  conclusion  is  that  most  of  the  information 
about  the  pixel  gray  level  under  estimation  resides  on  its 
closest  neighbors.   For  this  reason,  the  hybrid  filter 
presents  the  best  performance,  because  it  makes  use  of  all 
the  observations  in  the  neighborhood  and  also  those  farther 
observations  used  by  the  recursive  filter.   Experiments 
with  real  life  pictures  (figures  5.12  and  5.13)  show  the 
ability  of  the  hybrid  filter  in  smoothing  out  observation 
noise  in  pictures  with  signal-to-noise  ratio  as  low  as 
-3  dB.   The  processing  gain  for  this  case  was  11.1  dB,  which 
compares  favorably  with  previous  results. 
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For  images  sequenced  in  time,  the  three-dimensional 
recursive  and  hybrid  filters  were  developed  in  order  to 
exploit  the  correlation  in  time.   The  recursive  filter  is 
adequate  for  the  cases  where  the  scene  is  the  same  during 
several  frames,  otherwise  the  exploitation  of  the  time 
correlation  is  not  advantageous,  due  to  the  filter  transient 
at  the  first  few  frames.   Its  structure  is  intuitively 
appealing,  since  it  reduces  to  the  one-dimensional  Kalman 
filter  for  the  case  of  no  spatial  correlation,  and  to  a  two- 
dimensional  recursive  filter  for  the  case  of  no  correlation 
in  time.   Numerical  results  show  that  the  use  of  time  corre- 
lation can  be  quite  advantageous  for  the  case  of  time 
correlation  greater  than  spatial  correlation  (error  variance 
is  reduced  by  33%) ,  but  it  is  not  so  advantageous  for  the 
opposite  case  (error  variance  is  reduced  by.  6%)  .   The  hybrid 
filter  is  adequate  for  the  case  where  the  scene  changes  at 
every  two  frames,  since  it  makes  use  of  only  the  previous 
and  the  present  frames.   Of  course,  it  can  also  be  designed 
for  the  case  of  a  fixed  scenario  (see  Chapter  V)  in  which 
case  it  improves  the  performance  of  the  recursive  filter. 

To  detect  targets  from  cluttered  background  images  a 
likelihood  ratio  was  constructed  which  uses  the  observation 
of  the  pixel  and  the  background  prediction  for  the  pixel 
gray  level.   A  quite  general  situation  was  considered. 
The  target  and  background  are  considered  as  two  kinds  of 
textures.   The  image  is  modeled  as  a  weighted  combination  of 
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three  additive  components:   target,  background  and  obser- 
vation noise.   Therefore,  it  can  accommodate  the  case  of 
infrared  pictures  and  also  visible  pictures,  where  there  is 
a  replacement  process,  target  or  background.   The  prediction 
of  the  background  is  given  by  the  recursive  and  hybrid  filter 
or,  also,  by  a  non-recursive  filter.   The  likelihood  ratio 
was  particularized  for  four  special  cases  of  interest,  which 
had  their  performance  analyzed.   The  likelihood  ratio  enhanced 
the  target-to-background  ratio.   The  target  is  extracted  by 
threshold  detection.   The  threshold  is  chosen  in  order  to 
maintain  the  probability  of  false  alarm  below  some  desired 
value.   The  performance  of  the  detection  process  is  highly 
related  to  the  background-to-observation  noise  ratio,  since 
this  ratio  is  responsible  for  the  background  prediction. 

The  target  detection  and  tracking  problem  was  simulated 
using  comptuer  generated  images  with  characteristics  similar 
to  those  of  real  life  images  used  in  the  other  experiments. 
Both  three-dimensional  recursive  and  hybrid  predictors  were 
used  and  compared.   The  likelihood  ratio,  using  both  filters, 
was  able  to  detect  a  target  at  target-to-background  ratios 
as  low  as  -3  dB.   The  processing  gain  was  around  19  dB ,  for 
the  recursive  filter,  and  around  12.5  dB  for  the  hybrid 
filter.   The  recursive  filter  presented  better  performance, 
as  should  be  expected  from  the  results  of  Tables  5.3  through 
5.5,  where  its  prediction  is  better  than  that  of  the  hybrid 
filter,  for  the  case  of  high  background-to-noise  ratio. 
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B.   SUGGESTIONS  FOR  FUTURE  RESEARCH 

Many  pictures  are  clearly  non-homogeneous,  therefore 
the  development  of  space-varying  models  is  the  natural 
way  to  proceed  in  order  to  improve  the  model  suggested  in 
this  research.   The  recursive  techniques  are  directly 
amenable  to  the  analysis  of  space-varying  models.   The 
reliance  on  "a  priori"  information  is  a  very  important 
area.   The  method  of  parameter  identification  suggested 
here  is  applicable  for  the  case  of  constant  parameters  for 
the  whole  frame.   In  the  non-homogeneous  case  such  identi- 
fication has  to  be  accomplished  pixel  by  pixel,  in  order 
to  improve  the  robustness  of  this  statistical  approach 
in  face  of  modeling  errors. 

Due  to  the  enormous  computational  load  of  the  optimum 
multi-dimensional  recursive  filters,  and  also  because  the  • 
observations  far  away  carry  negligible  information  about 
the  pixel  to  be  estimated,  the  natural  way  is  to  look  for 
sub-optimal  recursive  filters  that  require  less  computation. 

The  hybrid  filters  suggested  here  can  be  equally  applied 
with  other  recursive  filters,  as,  for  example,  the  reduced 
dimension  filters  proposed  by  Panda  and  Kak  [9]  and  Woods 
and  Radiwan  [11] . 

The  decision  rule  suggested  here  is  directed  to  the 
detection  of  targets  pixel  by  pixel,  in  order  to  be  indepen- 
dent of  the  target  shape.   The  choice  of  the  threshold  was 
based  on  the  probability  of  a  false  alarm  of  each  isolated 
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decision.   Since  a  wrong  decision  necessarily  affects  the 
next,  a  question  arises  concerning  how  to  vary  the  threshold 
in  order  to  keep  the  false  alarm  probability  below  the 
desired  value. 
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